[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:30 dist-test-slave-77v9 krb5kdc[18207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:30 dist-test-slave-77v9 krb5kdc[18207](info): set up 2 sockets
Aug 11 22:14:30 dist-test-slave-77v9 krb5kdc[18207](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:14:31.736763 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:31 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:31.898003 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43499
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:43499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:32.018481 18221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:32.018862 18221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:32.018940 18221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:32.023218 18221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:32.023355 18221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:32.023392 18221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:32.023432 18221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:32.023535 18221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:32.029712 18221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:43499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:32.031034 18221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:32.032199 18221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:32.041987 18228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.041993 18231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.042001 18227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:32.042344 18221 server_base.cc:1034] running on GCE node
I20260811 22:14:32.043169 18221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:32.044519 18221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:32.045728 18221 hybrid_clock.cc:648] HybridClock initialized: now 1786486472045746 us; error 92 us; skew 500 ppm
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:32.049763 18221 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:32.051221 18221 webserver.cc:460] Webserver started at http://127.17.198.62:46253/ using document root <none> and password file <none>
I20260811 22:14:32.051903 18221 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:32.051978 18221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:32.052227 18221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:32.054703 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "2d41726202d14ea499f0716dc50c940f"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "432672116e1eaa8327477741568b4f24"
server_key_iv: "45cf21aab8474babaf8edf39e5dd88c0"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.055378 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "2d41726202d14ea499f0716dc50c940f"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "432672116e1eaa8327477741568b4f24"
server_key_iv: "45cf21aab8474babaf8edf39e5dd88c0"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.060643 18221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 22:14:32.063994 18239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.065784 18221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:32.065940 18221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "2d41726202d14ea499f0716dc50c940f"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "432672116e1eaa8327477741568b4f24"
server_key_iv: "45cf21aab8474babaf8edf39e5dd88c0"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.066067 18221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:32.085990 18221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:32.090847 18221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:32.091059 18221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:32.100344 18221 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:43499
I20260811 22:14:32.100370 18295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:43499 every 8 connection(s)
I20260811 22:14:32.101677 18221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:32.105113 18296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:32.108024 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18221
I20260811 22:14:32.108240 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:32.110162 18200 external_mini_cluster.cc:1428] Setting key 690c583b443480a90d6d5d6b7ca1650e
I20260811 22:14:32.113669 18296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: Bootstrap starting.
I20260811 22:14:32.116648 18296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:32.117630 18296 log.cc:824] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:32.120059 18296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: No bootstrap required, opened a new log
I20260811 22:14:32.124735 18296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } }
I20260811 22:14:32.125110 18296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:32.125170 18296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41726202d14ea499f0716dc50c940f, State: Initialized, Role: FOLLOWER
I20260811 22:14:32.125717 18296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } }
I20260811 22:14:32.125897 18296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:32.126024 18296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:32.126187 18296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:32.127404 18296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } }
I20260811 22:14:32.127840 18296 leader_election.cc:302] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d41726202d14ea499f0716dc50c940f; no voters:
I20260811 22:14:32.128228 18296 leader_election.cc:288] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:32.128384 18301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:32.128738 18301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [term 1 LEADER]: Becoming Leader. State: Replica: 2d41726202d14ea499f0716dc50c940f, State: Running, Role: LEADER
I20260811 22:14:32.129197 18301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } }
I20260811 22:14:32.129881 18296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:32.131165 18302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d41726202d14ea499f0716dc50c940f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } } }
I20260811 22:14:32.131342 18302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:32.131822 18303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d41726202d14ea499f0716dc50c940f. Latest consensus state: current_term: 1 leader_uuid: "2d41726202d14ea499f0716dc50c940f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d41726202d14ea499f0716dc50c940f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43499 } } }
I20260811 22:14:32.132057 18303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f [sys.catalog]: This master's current role is: LEADER
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
W20260811 22:14:32.136016 18318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:32.136149 18318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:32.136292 18311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:32.137614 18311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:32.142550 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:32.114741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.17.198.62:43499)
0811 22:14:32.115399 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:32.115435 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:32.116646 (+ 1211us) server_negotiation.cc:407] Connection header received
0811 22:14:32.120726 (+ 4080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:32.120749 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:32.121239 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:32.121628 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:32.124126 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.125807 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.127620 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.128239 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.128463 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:32.134874 (+ 6411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:32.134882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:32.134900 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:32.134942 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:32.137720 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.138610 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.138617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.138623 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.138789 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.139292 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.139297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.139299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.140098 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:32.140361 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:32.141082 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 22:14:32.141416 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":225,"threads_started":1}
I20260811 22:14:32.146142 18311 catalog_manager.cc:1347] Generated new cluster ID: bd46d51e4b2749f29fd77cc199128bf7
I20260811 22:14:32.146315 18311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:32.167106 18311 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:32.168494 18311 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:32.184301 18311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d41726202d14ea499f0716dc50c940f: Generated new TSK 0
I20260811 22:14:32.185330 18311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:32.230237 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:32.383565 18325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:32.383912 18325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:32.383962 18325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:32.388470 18325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:32.388578 18325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:32.388649 18325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:32.394793 18325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:32.395989 18325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:32.397083 18325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:32.409276 18330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.409241 18333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.409241 18331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:32.409343 18325 server_base.cc:1034] running on GCE node
I20260811 22:14:32.410071 18325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:32.410862 18325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:32.412055 18325 hybrid_clock.cc:648] HybridClock initialized: now 1786486472412028 us; error 41 us; skew 500 ppm
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:32.415678 18325 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:32.416648 18325 webserver.cc:460] Webserver started at http://127.17.198.1:41957/ using document root <none> and password file <none>
I20260811 22:14:32.417330 18325 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:32.417399 18325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:32.417582 18325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:32.419848 18325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "5341bc6427ae4416a8064a4cf7d9ceaf"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "8befe8db68cbc8c6eca6d4d6fca97df6"
server_key_iv: "4ae738ebe9d06a1b7ee444261d2774a3"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.420550 18325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "5341bc6427ae4416a8064a4cf7d9ceaf"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "8befe8db68cbc8c6eca6d4d6fca97df6"
server_key_iv: "4ae738ebe9d06a1b7ee444261d2774a3"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.424932 18325 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:32.429245 18340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.431950 18325 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:14:32.432181 18325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "5341bc6427ae4416a8064a4cf7d9ceaf"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "8befe8db68cbc8c6eca6d4d6fca97df6"
server_key_iv: "4ae738ebe9d06a1b7ee444261d2774a3"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.432459 18325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:32.453085 18325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:32.458425 18325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:32.458794 18325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:32.460137 18325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:32.461426 18325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:32.461515 18325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.461599 18325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:32.461637 18325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.476011 18325 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:42785
I20260811 22:14:32.476094 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:42785 every 8 connection(s)
I20260811 22:14:32.477265 18325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:32.481372 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18325
I20260811 22:14:32.481537 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:32.481832 18200 external_mini_cluster.cc:1428] Setting key a1c5c2f142e1e2ecc68cfefcd68357dc
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:32.496395 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:32.479624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:52339 (local address 127.17.198.62:43499)
0811 22:14:32.479883 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:32.479889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:32.481291 (+ 1402us) server_negotiation.cc:407] Connection header received
0811 22:14:32.482423 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:32.482429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:32.482508 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:32.482607 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:32.484094 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.485010 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.487121 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.487412 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.487523 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:32.491514 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:32.491518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:32.491520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:32.491552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:32.493527 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.494440 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.494443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.494444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.494547 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.495059 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.495062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.495064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.495291 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:32.495389 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:32.496102 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 22:14:32.496250 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:14:32.497419 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:32.480157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43499 (local address 127.17.198.1:52339)
0811 22:14:32.481105 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:32.481160 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:32.482184 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:32.482835 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:32.482843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:32.483266 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:32.483925 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:32.483941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.485176 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.485179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:32.486730 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:32.486745 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.487620 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.487628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:32.487913 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:32.488750 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:32.488782 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:32.491179 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:32.493745 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:32.493760 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:32.493784 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:32.494291 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:32.494672 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:32.494677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:32.494679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:32.494929 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:32.495436 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:32.495441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:32.495742 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 22:14:32.496015 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:14:32.496351 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":161,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:14:32.499106 18464 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43499
I20260811 22:14:32.499418 18464 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:32.500223 18464 heartbeater.cc:511] Master 127.17.198.62:43499 requested a full tablet report, sending...
I20260811 22:14:32.502485 18259 ts_manager.cc:194] Registered new tserver with Master: 5341bc6427ae4416a8064a4cf7d9ceaf (127.17.198.1:42785)
I20260811 22:14:32.504437 18259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:52339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:32.521697 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:32.669566 18469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:32.669929 18469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:32.669986 18469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:32.675177 18469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:32.675299 18469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:32.675374 18469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:32.681771 18469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:32.682982 18469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:32.684333 18469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:32.695709 18477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.695751 18475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:32.695838 18469 server_base.cc:1034] running on GCE node
W20260811 22:14:32.695709 18474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:32.696444 18469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:32.697279 18469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:32.698516 18469 hybrid_clock.cc:648] HybridClock initialized: now 1786486472698486 us; error 42 us; skew 500 ppm
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:32.702219 18469 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:32.703325 18469 webserver.cc:460] Webserver started at http://127.17.198.2:35583/ using document root <none> and password file <none>
I20260811 22:14:32.704002 18469 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:32.704111 18469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:32.704377 18469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:32.706796 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "ff50baada7484ff183ac6121fa5b3259"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "25cfb532b31acb96b70f03127e37cb17"
server_key_iv: "c5ccdff935339a684708972eef100ea4"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.707485 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "ff50baada7484ff183ac6121fa5b3259"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "25cfb532b31acb96b70f03127e37cb17"
server_key_iv: "c5ccdff935339a684708972eef100ea4"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.712018 18469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:14:32.715099 18484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.716662 18469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:32.716879 18469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "ff50baada7484ff183ac6121fa5b3259"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "25cfb532b31acb96b70f03127e37cb17"
server_key_iv: "c5ccdff935339a684708972eef100ea4"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.717031 18469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:32.731066 18469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:32.735473 18469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:32.735809 18469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:32.736662 18469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:32.737777 18469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:32.737846 18469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.737895 18469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:32.737911 18469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.751569 18469 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43495
I20260811 22:14:32.751596 18613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43495 every 8 connection(s)
I20260811 22:14:32.752789 18469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:32.762293 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18469
I20260811 22:14:32.762473 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:32.762773 18200 external_mini_cluster.cc:1428] Setting key 0fe59f189930e1bc9d252938541de13d
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:32.769745 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:32.755237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:59527 (local address 127.17.198.62:43499)
0811 22:14:32.755416 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:32.755422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:32.756492 (+ 1070us) server_negotiation.cc:407] Connection header received
0811 22:14:32.757973 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:32.757977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:32.758036 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:32.758135 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:32.759698 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.760384 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.761711 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.762020 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.762153 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:32.765609 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:32.765613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:32.765621 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:32.765648 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:32.767325 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.767980 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.767984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.767985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.768067 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:32.768461 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:32.768465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:32.768466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:32.768685 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:32.768846 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:32.769429 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 22:14:32.769574 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:14:32.770798 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:32.755751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43499 (local address 127.17.198.2:59527)
0811 22:14:32.756297 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:32.756354 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:32.757643 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:32.758389 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:32.758398 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:32.758955 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:32.759498 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:32.759514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.760534 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.760537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:32.761572 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:32.761583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:32.762251 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:32.762257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:32.762521 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:32.763555 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:32.763577 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:32.765411 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:32.767485 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:32.767493 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:32.767513 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:32.767852 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:32.768171 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:32.768174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:32.768176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:32.768351 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:32.768856 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:32.768867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:32.769154 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 22:14:32.769423 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 22:14:32.769821 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":142,"threads_started":1}
I20260811 22:14:32.772215 18614 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43499
I20260811 22:14:32.772557 18614 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:32.773316 18614 heartbeater.cc:511] Master 127.17.198.62:43499 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:14:32.774835 18259 ts_manager.cc:194] Registered new tserver with Master: ff50baada7484ff183ac6121fa5b3259 (127.17.198.2:43495)
I20260811 22:14:32.775696 18259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:59527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:32.797947 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:32.926705 18620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:32.927275 18620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:32.927331 18620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:32.932286 18620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:32.932384 18620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:32.932512 18620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:32.938767 18620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43499
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:32.940060 18620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:32.941251 18620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:32.954519 18628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.954507 18625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:32.954507 18626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:32.954602 18620 server_base.cc:1034] running on GCE node
I20260811 22:14:32.955380 18620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:32.956297 18620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:32.957530 18620 hybrid_clock.cc:648] HybridClock initialized: now 1786486472957514 us; error 45 us; skew 500 ppm
Aug 11 22:14:32 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:32.961452 18620 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:32.962667 18620 webserver.cc:460] Webserver started at http://127.17.198.3:40405/ using document root <none> and password file <none>
I20260811 22:14:32.963361 18620 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:32.963440 18620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:32.963646 18620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:32.966406 18620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "c1e5f783546842e8a6a486088223c04b"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "7e1fdd67040a0a20562dea60a004ba7a"
server_key_iv: "9d846c2adf47abb9502c27afa0925503"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.967110 18620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "c1e5f783546842e8a6a486088223c04b"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "7e1fdd67040a0a20562dea60a004ba7a"
server_key_iv: "9d846c2adf47abb9502c27afa0925503"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.972486 18620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:14:32.975950 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:32.977773 18620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:32.978057 18620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "c1e5f783546842e8a6a486088223c04b"
format_stamp: "Formatted at 2026-08-11 22:14:32 on dist-test-slave-77v9"
server_key: "7e1fdd67040a0a20562dea60a004ba7a"
server_key_iv: "9d846c2adf47abb9502c27afa0925503"
server_key_version: "encryptionkey@0"
I20260811 22:14:32.978219 18620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:32.997345 18620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:33.001734 18620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:33.002031 18620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:33.002846 18620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:33.003892 18620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:33.003952 18620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:33.003998 18620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:33.004016 18620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:33.019343 18620 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:43025
I20260811 22:14:33.019446 18757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:43025 every 8 connection(s)
I20260811 22:14:33.020722 18620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:33.027443 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18620
I20260811 22:14:33.027598 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:33.027884 18200 external_mini_cluster.cc:1428] Setting key 5435f74d2e20200a7c07c04a8a2e9050
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:33.040237 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:33.023910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:57235 (local address 127.17.198.62:43499)
0811 22:14:33.024089 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:33.024094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:33.025315 (+ 1221us) server_negotiation.cc:407] Connection header received
0811 22:14:33.027180 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:33.027186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:33.027265 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:33.027395 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:33.029487 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.030084 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.031945 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.032202 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.032314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:33.035526 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:33.035530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:33.035532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:33.035566 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:33.037438 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.038136 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.038139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.038140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.038239 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.038764 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.038768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.038770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.039005 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:33.039119 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:33.039931 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 22:14:33.040093 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:14:33.041463 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:33.024343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43499 (local address 127.17.198.3:57235)
0811 22:14:33.025119 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:33.025173 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:33.026741 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:33.027675 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:33.027690 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:33.028390 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:33.029299 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:33.029322 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.030293 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.030297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:33.031767 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:33.031780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.032397 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.032403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:33.032604 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:33.033430 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:33.033457 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:33.035260 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:33.037625 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:33.037632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:33.037650 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:33.037991 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:33.038365 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:33.038368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:33.038369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:33.038617 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:33.039158 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:33.039167 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:33.039572 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 22:14:33.039828 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 22:14:33.040160 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":190,"threads_started":1}
I20260811 22:14:33.042924 18758 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43499
I20260811 22:14:33.043380 18758 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:33.044143 18758 heartbeater.cc:511] Master 127.17.198.62:43499 requested a full tablet report, sending...
I20260811 22:14:33.045522 18258 ts_manager.cc:194] Registered new tserver with Master: c1e5f783546842e8a6a486088223c04b (127.17.198.3:43025)
I20260811 22:14:33.046330 18258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:57235
I20260811 22:14:33.056357 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486473, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486473, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:33.089725 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:33.077819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.17.198.1:42785)
0811 22:14:33.078269 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:33.078276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:33.078307 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:14:33.078384 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:33.078389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:33.078613 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:33.078769 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:33.079556 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.080573 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.081725 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.082291 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.082409 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:33.085292 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:33.085346 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:33.085362 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:33.085397 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:33.087316 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.087854 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.087860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.087861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.087989 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.088258 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.088260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.088261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.088528 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:33.088692 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:33.089284 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:14:33.089508 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":189,"threads_started":1}
W20260811 22:14:33.091284 18371 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:33.129227 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:33.118529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.17.198.1:42785)
0811 22:14:33.118723 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:33.118726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:33.118747 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:14:33.118952 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:33.118955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:33.119002 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:33.119081 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:33.120153 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.120882 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.122002 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:33.122252 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:33.122348 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:33.125300 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:33.125305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:33.125307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:33.125351 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:33.127221 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.127763 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.127766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.127767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.127867 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:33.128180 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:33.128182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:33.128184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:33.128376 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:33.128500 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:33.128896 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:14:33.129037 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:14:33.130612 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18325
I20260811 22:14:33.139326 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18469
I20260811 22:14:33.150918 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18620
I20260811 22:14:33.163784 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18221
2026-08-11T22:14:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2682 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18774](info): set up 2 sockets
Aug 11 22:14:33 dist-test-slave-77v9 krb5kdc[18774](info): commencing operation
krb5kdc: starting...
W20260811 22:14:34.370610 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:34 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:34.521987 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45789
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:45789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:34.645092 18789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:34.645495 18789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:34.645586 18789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:34.650259 18789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:34.650385 18789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:34.650425 18789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:34.650475 18789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:34.650534 18789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:34.656430 18789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:45789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:34.657886 18789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:34.659109 18789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:34.669682 18797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:34.669687 18794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:34.669747 18789 server_base.cc:1034] running on GCE node
W20260811 22:14:34.669757 18795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:34.670691 18789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:34.672147 18789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:34.673401 18789 hybrid_clock.cc:648] HybridClock initialized: now 1786486474673371 us; error 50 us; skew 500 ppm
Aug 11 22:14:34 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486474, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:34.677284 18789 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:34.678478 18789 webserver.cc:460] Webserver started at http://127.17.198.62:38007/ using document root <none> and password file <none>
I20260811 22:14:34.679190 18789 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:34.679308 18789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:34.679610 18789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:34.682322 18789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "1e88156a7008409d9512c47a0d0fc543"
format_stamp: "Formatted at 2026-08-11 22:14:34 on dist-test-slave-77v9"
server_key: "473a0b6725580c101463eef79ace4db3"
server_key_iv: "1ecf37134007dcdf63889d082825ac8f"
server_key_version: "encryptionkey@0"
I20260811 22:14:34.683036 18789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "1e88156a7008409d9512c47a0d0fc543"
format_stamp: "Formatted at 2026-08-11 22:14:34 on dist-test-slave-77v9"
server_key: "473a0b6725580c101463eef79ace4db3"
server_key_iv: "1ecf37134007dcdf63889d082825ac8f"
server_key_version: "encryptionkey@0"
I20260811 22:14:34.687740 18789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:34.691013 18804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:34.692695 18789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:34.693032 18789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "1e88156a7008409d9512c47a0d0fc543"
format_stamp: "Formatted at 2026-08-11 22:14:34 on dist-test-slave-77v9"
server_key: "473a0b6725580c101463eef79ace4db3"
server_key_iv: "1ecf37134007dcdf63889d082825ac8f"
server_key_version: "encryptionkey@0"
I20260811 22:14:34.693204 18789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:34.730011 18789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:34.734220 18789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:34.734504 18789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:34.743194 18789 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:45789
I20260811 22:14:34.743249 18863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:45789 every 8 connection(s)
I20260811 22:14:34.744349 18789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:34.748176 18864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:34.751456 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18789
I20260811 22:14:34.751632 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:34.751938 18200 external_mini_cluster.cc:1428] Setting key 6d10214d0f72263a3e49c4ddb0e46799
I20260811 22:14:34.755306 18864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: Bootstrap starting.
I20260811 22:14:34.758592 18864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:34.759776 18864 log.cc:824] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:34.762277 18864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: No bootstrap required, opened a new log
Aug 11 22:14:34 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:34.765720 18864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } }
I20260811 22:14:34.766045 18864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:34.766134 18864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e88156a7008409d9512c47a0d0fc543, State: Initialized, Role: FOLLOWER
I20260811 22:14:34.766683 18864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } }
I20260811 22:14:34.766852 18864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:34.766952 18864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:34.767076 18864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:34.768390 18864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } }
I20260811 22:14:34.768850 18864 leader_election.cc:302] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e88156a7008409d9512c47a0d0fc543; no voters:
I20260811 22:14:34.769291 18864 leader_election.cc:288] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:34.769510 18869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:34.769860 18869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [term 1 LEADER]: Becoming Leader. State: Replica: 1e88156a7008409d9512c47a0d0fc543, State: Running, Role: LEADER
I20260811 22:14:34.770293 18869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } }
I20260811 22:14:34.770658 18864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:34.770684 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:34.753631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.17.198.62:45789)
0811 22:14:34.754140 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:34.754158 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:34.754199 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:14:34.754910 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:34.754928 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:34.755302 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:34.755695 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:34.756795 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:34.758154 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:34.759615 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:34.760299 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:34.760520 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:34.763669 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:34.763675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:34.763691 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:34.763724 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:34.766808 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:34.767523 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:34.767529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:34.767534 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:34.767672 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:34.768034 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:34.768038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:34.768039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:34.768637 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:34.768914 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:34.769208 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:14:34.769596 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":164,"threads_started":1}
I20260811 22:14:34.772296 18870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e88156a7008409d9512c47a0d0fc543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } } }
I20260811 22:14:34.772307 18871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e88156a7008409d9512c47a0d0fc543. Latest consensus state: current_term: 1 leader_uuid: "1e88156a7008409d9512c47a0d0fc543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e88156a7008409d9512c47a0d0fc543" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45789 } } }
I20260811 22:14:34.772461 18870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:34.772485 18871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543 [sys.catalog]: This master's current role is: LEADER
W20260811 22:14:34.776254 18884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:34.776378 18884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:34.776630 18881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:34.777448 18881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:34.784576 18881 catalog_manager.cc:1347] Generated new cluster ID: f8260b26aac34eb980f1d0e166c3f77d
I20260811 22:14:34.784708 18881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:34.792883 18881 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:34.794036 18881 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:34.801601 18881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e88156a7008409d9512c47a0d0fc543: Generated new TSK 0
I20260811 22:14:34.802435 18881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:34.841394 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:34.977955 18891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:34.978288 18891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:34.978382 18891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:34.983516 18891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:34.983623 18891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:34.983697 18891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:34.990528 18891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:34.991845 18891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:34.993121 18891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:35.005681 18897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.005663 18896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.005668 18899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:35.006028 18891 server_base.cc:1034] running on GCE node
I20260811 22:14:35.006573 18891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:35.007309 18891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:35.008503 18891 hybrid_clock.cc:648] HybridClock initialized: now 1786486475008486 us; error 39 us; skew 500 ppm
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:35.012284 18891 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:35.013496 18891 webserver.cc:460] Webserver started at http://127.17.198.1:45737/ using document root <none> and password file <none>
I20260811 22:14:35.014159 18891 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:35.014268 18891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:35.014534 18891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:35.017119 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "feef14da689c444b95d1fcc36d0e7284"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "0a58e1e14f841a402db5c1cf16f417f8"
server_key_iv: "4b68c7a857d5d07dca3abd259c883f7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.017859 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "feef14da689c444b95d1fcc36d0e7284"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "0a58e1e14f841a402db5c1cf16f417f8"
server_key_iv: "4b68c7a857d5d07dca3abd259c883f7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.022508 18891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:35.025523 18909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.027231 18891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:35.027462 18891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "feef14da689c444b95d1fcc36d0e7284"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "0a58e1e14f841a402db5c1cf16f417f8"
server_key_iv: "4b68c7a857d5d07dca3abd259c883f7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.027624 18891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:35.056495 18891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:35.060452 18891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:35.060716 18891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:35.061466 18891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:35.062463 18891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:35.062551 18891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.062626 18891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:35.062680 18891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.073587 18891 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:46237
I20260811 22:14:35.073611 19033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:46237 every 8 connection(s)
I20260811 22:14:35.074669 18891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:35.081609 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 18891
I20260811 22:14:35.081789 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:35.082166 18200 external_mini_cluster.cc:1428] Setting key 2072cbcb65ae306a079febe53cde3dd2
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:35.092156 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.076931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:58089 (local address 127.17.198.62:45789)
0811 22:14:35.077098 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.077102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.078248 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 22:14:35.079785 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.079790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.079862 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.079987 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.081446 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.082082 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.083518 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.083836 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.083966 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.087141 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.087148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.087154 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.087192 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.089239 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.090014 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.090020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.090023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.090134 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.090544 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.090550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.090554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.090892 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.091009 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.091713 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.091912 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:14:35.093152 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.077406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45789 (local address 127.17.198.1:58089)
0811 22:14:35.077957 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.078019 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.079514 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.080194 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.080202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.080630 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:35.081265 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.081282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.082238 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.082241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.083357 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.083368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.084026 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.084030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.084217 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.084944 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:35.084963 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:35.086889 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:35.089419 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.089431 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.089456 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.089850 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.090247 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.090250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.090251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.090419 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.091098 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:35.091105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:35.091413 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.091608 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.091943 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:35.094906 19034 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45789
I20260811 22:14:35.095191 19034 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:35.095734 19034 heartbeater.cc:511] Master 127.17.198.62:45789 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:14:35.097860 18823 ts_manager.cc:194] Registered new tserver with Master: feef14da689c444b95d1fcc36d0e7284 (127.17.198.1:46237)
I20260811 22:14:35.099663 18823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:58089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:35.117143 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:35.243415 19039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:35.243736 19039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:35.243784 19039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:35.248080 19039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:35.248181 19039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:35.248250 19039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:35.254077 19039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:35.255443 19039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:35.256498 19039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:35.267678 19044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.267683 19047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.267725 19045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:35.268090 19039 server_base.cc:1034] running on GCE node
I20260811 22:14:35.268591 19039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:35.269376 19039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:35.270671 19039 hybrid_clock.cc:648] HybridClock initialized: now 1786486475270653 us; error 42 us; skew 500 ppm
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:35.274159 19039 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:35.275214 19039 webserver.cc:460] Webserver started at http://127.17.198.2:46373/ using document root <none> and password file <none>
I20260811 22:14:35.275828 19039 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:35.275929 19039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:35.276156 19039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:35.278542 19039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "7d369bd03946448097f077439289bf82"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "f3c5d66f3e96d2eb338981b03c499f2f"
server_key_iv: "01e195127b29e15fb0a6799d922f9ba6"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.279199 19039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "7d369bd03946448097f077439289bf82"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "f3c5d66f3e96d2eb338981b03c499f2f"
server_key_iv: "01e195127b29e15fb0a6799d922f9ba6"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.284255 19039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:14:35.287578 19056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.289348 19039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:14:35.289534 19039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "7d369bd03946448097f077439289bf82"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "f3c5d66f3e96d2eb338981b03c499f2f"
server_key_iv: "01e195127b29e15fb0a6799d922f9ba6"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.289690 19039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:35.305810 19039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:35.309615 19039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:35.309859 19039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:35.310504 19039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:35.311424 19039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:35.311497 19039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.311573 19039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:35.311626 19039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.321370 19039 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:41527
I20260811 22:14:35.321393 19181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:41527 every 8 connection(s)
I20260811 22:14:35.322404 19039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:35.326226 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19039
I20260811 22:14:35.326385 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:35.326682 18200 external_mini_cluster.cc:1428] Setting key d9effc4514bcf8c119a3ab9a1663b505
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:14:35.341724 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.324471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:55041 (local address 127.17.198.62:45789)
0811 22:14:35.324626 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.324629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.325694 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 22:14:35.327408 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.327412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.327479 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.327578 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.329247 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.329857 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.332099 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.332594 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.332738 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.336942 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.336953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.336958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.336998 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.338721 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.339623 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.339627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.339628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.339728 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.340296 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.340299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.340300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.340524 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.340610 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.341451 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.341590 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:14:35.342667 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.324947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45789 (local address 127.17.198.2:55041)
0811 22:14:35.325491 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.325552 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.327124 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.327787 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.327796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.328408 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:35.329067 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.329086 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.330043 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.330046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.331921 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.331937 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.333050 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.333058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.333364 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.334164 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:35.334195 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:35.336563 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:35.338922 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.338935 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.338963 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.339469 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.339976 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.339979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.339980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.340171 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.340636 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:35.340643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:35.341134 (+ 491us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.341320 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.341644 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:14:35.344406 19182 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45789
I20260811 22:14:35.344700 19182 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:35.345358 19182 heartbeater.cc:511] Master 127.17.198.62:45789 requested a full tablet report, sending...
I20260811 22:14:35.346812 18823 ts_manager.cc:194] Registered new tserver with Master: 7d369bd03946448097f077439289bf82 (127.17.198.2:41527)
I20260811 22:14:35.347512 18823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:55041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:35.365132 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:35.516984 19188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:35.517328 19188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:35.517380 19188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:35.522332 19188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:35.522454 19188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:35.522533 19188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:35.529897 19188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:35.531232 19188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:35.532383 19188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:35.543798 19193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.543715 19197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:35.543731 19195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:35.544030 19188 server_base.cc:1034] running on GCE node
I20260811 22:14:35.544600 19188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:35.545414 19188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:35.546604 19188 hybrid_clock.cc:648] HybridClock initialized: now 1786486475546535 us; error 107 us; skew 500 ppm
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:35.550139 19188 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:35.551167 19188 webserver.cc:460] Webserver started at http://127.17.198.3:45839/ using document root <none> and password file <none>
I20260811 22:14:35.551844 19188 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:35.551918 19188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:35.552103 19188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:35.554773 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "0b91e34238f04b06baa97240e7b69754"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "d4f1db60cc54099edbae589f49c28172"
server_key_iv: "00106df3303bb53fecafdefacdc80e43"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.555419 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "0b91e34238f04b06baa97240e7b69754"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "d4f1db60cc54099edbae589f49c28172"
server_key_iv: "00106df3303bb53fecafdefacdc80e43"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.559788 19188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:35.563277 19204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.565137 19188 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:35.565304 19188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "0b91e34238f04b06baa97240e7b69754"
format_stamp: "Formatted at 2026-08-11 22:14:35 on dist-test-slave-77v9"
server_key: "d4f1db60cc54099edbae589f49c28172"
server_key_iv: "00106df3303bb53fecafdefacdc80e43"
server_key_version: "encryptionkey@0"
I20260811 22:14:35.565467 19188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:35.596343 19188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:35.600667 19188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:35.601054 19188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:35.601708 19188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:35.602701 19188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:35.602761 19188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.602847 19188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:35.602890 19188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:35.618809 19188 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:35055
I20260811 22:14:35.619134 19328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:35055 every 8 connection(s)
I20260811 22:14:35.620105 19188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:35.624801 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19188
I20260811 22:14:35.624989 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:35.625286 18200 external_mini_cluster.cc:1428] Setting key fedbf14ae67e23b4f18472b563e8ab58
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:35.638149 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.622417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:44433 (local address 127.17.198.62:45789)
0811 22:14:35.622587 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.622591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.623567 (+ 976us) server_negotiation.cc:407] Connection header received
0811 22:14:35.625588 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.625592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.625664 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.625791 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.627377 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.627969 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.630097 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.630343 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.630481 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.633774 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.633778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.633780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.633809 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.635654 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.636292 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.636298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.636305 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.636392 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.636818 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.636852 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.636854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.637080 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.637200 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.637803 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.637959 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:14:35.639410 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.622788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45789 (local address 127.17.198.3:44433)
0811 22:14:35.623374 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.623424 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.625271 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.626016 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.626024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.626568 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:35.627198 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.627215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.628116 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.628120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.629931 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.629949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.630522 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.630527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.630796 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.631641 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:35.631661 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:35.633539 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:35.635806 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.635814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.635835 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.636159 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.636513 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:35.636516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:35.636518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:35.636694 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:35.637200 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:35.637209 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:35.637523 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.637740 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.638161 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":138,"threads_started":1}
I20260811 22:14:35.640902 19329 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45789
I20260811 22:14:35.641379 19329 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:35.642067 19329 heartbeater.cc:511] Master 127.17.198.62:45789 requested a full tablet report, sending...
I20260811 22:14:35.643419 18823 ts_manager.cc:194] Registered new tserver with Master: 0b91e34238f04b06baa97240e7b69754 (127.17.198.3:35055)
I20260811 22:14:35.644179 18823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:44433
I20260811 22:14:35.652853 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:35.666055 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.657341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.17.198.62:45789)
0811 22:14:35.657504 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.657508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.657717 (+ 209us) server_negotiation.cc:407] Connection header received
0811 22:14:35.657837 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.657841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.657897 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.657987 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.658867 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.659442 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.660366 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.660604 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.660702 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.662418 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.662421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.662424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.662456 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.663909 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.664562 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.664565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.664566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.664658 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.665029 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.665032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.665033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.665297 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.665437 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.665737 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.665924 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:14:35.672498 18823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:35.675964 18823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:35.693113 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.687068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.17.198.1:46237)
0811 22:14:35.687800 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.687807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.687829 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:14:35.687907 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.687913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.688325 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.688486 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:35.688755 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.689906 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.691395 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692261 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692385 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692562 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.692789 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.692945 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":107,"threads_started":1}
I20260811 22:14:35.693152 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.686181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:41527 (local address 127.0.0.1:36482)
0811 22:14:35.687243 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.687258 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.687445 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.688197 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.688199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.688218 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:35.688421 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.688432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.690034 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.690036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.691207 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.691216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692646 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.692766 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692856 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.692927 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.692998 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":59,"spinlock_wait_cycles":1408,"thread_start_us":107,"threads_started":1}
I20260811 22:14:35.693428 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.686016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.17.198.3:35055)
0811 22:14:35.686769 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.686785 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.687187 (+ 402us) server_negotiation.cc:407] Connection header received
0811 22:14:35.687554 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.687560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.687825 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.687974 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:35.688542 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.689483 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.691437 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692392 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692486 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692637 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.693102 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.693212 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":310,"threads_started":1}
I20260811 22:14:35.693681 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.686881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46237 (local address 127.0.0.1:59332)
0811 22:14:35.687375 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.687391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.687484 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.688488 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.688492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.688515 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:35.688610 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.688617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.690037 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.690041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.691287 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.691296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692412 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.692555 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692600 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.692737 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.692968 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":62,"thread_start_us":118,"threads_started":1}
I20260811 22:14:35.693953 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.685955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:35055 (local address 127.0.0.1:39184)
0811 22:14:35.687003 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.687071 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.687335 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.688163 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.688170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.688199 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:35.688361 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.688368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.689652 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.689654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.691258 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.691271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692761 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.692918 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692943 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.693666 (+ 723us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.693780 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":91,"spinlock_wait_cycles":768,"thread_start_us":141,"threads_started":1}
I20260811 22:14:35.694401 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.686439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.17.198.2:41527)
0811 22:14:35.686969 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.686981 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.687360 (+ 379us) server_negotiation.cc:407] Connection header received
0811 22:14:35.687606 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.687612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.687829 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.687971 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:35.688563 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.689847 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.691342 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.692261 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.692385 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.692572 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.693555 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.694249 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":167,"threads_started":1}
I20260811 22:14:35.697268 18959 tablet_service.cc:1467] Processing CreateTablet for tablet bc8218c1ef74471aa002c66d5b84eada (DEFAULT_TABLE table=test-table [id=b6dd88b965404bd08c2b114566461df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:35.697597 19109 tablet_service.cc:1467] Processing CreateTablet for tablet bc8218c1ef74471aa002c66d5b84eada (DEFAULT_TABLE table=test-table [id=b6dd88b965404bd08c2b114566461df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:35.698601 18959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8218c1ef74471aa002c66d5b84eada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:35.698397 19258 tablet_service.cc:1467] Processing CreateTablet for tablet bc8218c1ef74471aa002c66d5b84eada (DEFAULT_TABLE table=test-table [id=b6dd88b965404bd08c2b114566461df4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:35.698932 19109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8218c1ef74471aa002c66d5b84eada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:35.699433 19258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8218c1ef74471aa002c66d5b84eada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:35.706650 19349 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Bootstrap starting.
I20260811 22:14:35.707602 19351 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Bootstrap starting.
I20260811 22:14:35.707603 19350 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Bootstrap starting.
I20260811 22:14:35.709959 19349 tablet_bootstrap.cc:654] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:35.710309 19351 tablet_bootstrap.cc:654] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:35.710737 19350 tablet_bootstrap.cc:654] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:35.711186 19349 log.cc:824] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:35.711416 19351 log.cc:824] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:35.711827 19350 log.cc:824] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:35.713706 19351 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: No bootstrap required, opened a new log
I20260811 22:14:35.713709 19349 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: No bootstrap required, opened a new log
I20260811 22:14:35.713960 19351 ts_tablet_manager.cc:1397] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:14:35.713961 19349 ts_tablet_manager.cc:1397] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:14:35.717715 19351 raft_consensus.cc:348] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.718084 19351 raft_consensus.cc:374] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:35.718159 19351 raft_consensus.cc:729] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b91e34238f04b06baa97240e7b69754, State: Initialized, Role: FOLLOWER
I20260811 22:14:35.718487 19350 tablet_bootstrap.cc:492] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: No bootstrap required, opened a new log
I20260811 22:14:35.718772 19350 ts_tablet_manager.cc:1397] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.001s
I20260811 22:14:35.718732 19351 consensus_queue.cc:260] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.719543 19329 heartbeater.cc:503] Master 127.17.198.62:45789 was elected leader, sending a full tablet report...
I20260811 22:14:35.719340 19349 raft_consensus.cc:348] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.719632 19349 raft_consensus.cc:374] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:35.719688 19349 raft_consensus.cc:729] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feef14da689c444b95d1fcc36d0e7284, State: Initialized, Role: FOLLOWER
I20260811 22:14:35.719981 19351 ts_tablet_manager.cc:1428] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:14:35.720201 19349 consensus_queue.cc:260] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.721284 19034 heartbeater.cc:503] Master 127.17.198.62:45789 was elected leader, sending a full tablet report...
I20260811 22:14:35.721381 19349 ts_tablet_manager.cc:1428] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:14:35.725132 19350 raft_consensus.cc:348] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.725567 19350 raft_consensus.cc:374] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:35.725646 19350 raft_consensus.cc:729] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d369bd03946448097f077439289bf82, State: Initialized, Role: FOLLOWER
I20260811 22:14:35.726080 19350 consensus_queue.cc:260] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.726842 19182 heartbeater.cc:503] Master 127.17.198.62:45789 was elected leader, sending a full tablet report...
I20260811 22:14:35.727253 19350 ts_tablet_manager.cc:1428] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260811 22:14:35.747072 19356 raft_consensus.cc:482] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:35.747390 19356 raft_consensus.cc:504] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.748935 19356 leader_election.cc:288] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers feef14da689c444b95d1fcc36d0e7284 (127.17.198.1:46237), 7d369bd03946448097f077439289bf82 (127.17.198.2:41527)
I20260811 22:14:35.754206 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.749920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:57851 (local address 127.17.198.2:41527)
0811 22:14:35.750071 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.750076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.750092 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:14:35.750148 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.750151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.750210 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.750294 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:35.750671 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.751469 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.753096 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.753561 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.753720 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.753808 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.753981 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.754072 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:14:35.754248 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.749254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:41527 (local address 127.17.198.3:57851)
0811 22:14:35.749911 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.749927 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.750022 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.750372 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.750375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.750395 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:35.750508 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.750515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.751600 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.751603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.752952 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.752961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.753713 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.753717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.753828 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.753866 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.753951 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.754042 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":2,"thread_start_us":202,"threads_started":1}
I20260811 22:14:35.754593 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.749387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:50043 (local address 127.17.198.1:46237)
0811 22:14:35.749529 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.749535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.749552 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:35.749742 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.749745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.749804 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.749889 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:35.750342 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.751037 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.752846 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.753617 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.753759 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.753826 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.754328 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.754438 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:14:35.754973 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.749201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46237 (local address 127.17.198.3:50043)
0811 22:14:35.749447 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:35.749466 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:35.749582 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:35.750042 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:35.750049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:35.750070 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:35.750204 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.750210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.751182 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.751186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.752651 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:35.752663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.753916 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.753923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:35.754047 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.754068 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:14:35.754143 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:14:35.754209 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":37}
I20260811 22:14:35.755240 19130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8218c1ef74471aa002c66d5b84eada" candidate_uuid: "0b91e34238f04b06baa97240e7b69754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d369bd03946448097f077439289bf82" is_pre_election: true
I20260811 22:14:35.755762 18982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8218c1ef74471aa002c66d5b84eada" candidate_uuid: "0b91e34238f04b06baa97240e7b69754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feef14da689c444b95d1fcc36d0e7284" is_pre_election: true
I20260811 22:14:35.755836 19130 raft_consensus.cc:2454] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b91e34238f04b06baa97240e7b69754 in term 0.
I20260811 22:14:35.756089 18982 raft_consensus.cc:2454] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b91e34238f04b06baa97240e7b69754 in term 0.
I20260811 22:14:35.756523 19205 leader_election.cc:302] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b91e34238f04b06baa97240e7b69754, 7d369bd03946448097f077439289bf82; no voters:
I20260811 22:14:35.757184 19356 raft_consensus.cc:2781] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:35.757294 19356 raft_consensus.cc:482] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:35.757355 19356 raft_consensus.cc:3037] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:35.758767 19356 raft_consensus.cc:504] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.759256 19356 leader_election.cc:288] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [CANDIDATE]: Term 1 election: Requested vote from peers feef14da689c444b95d1fcc36d0e7284 (127.17.198.1:46237), 7d369bd03946448097f077439289bf82 (127.17.198.2:41527)
I20260811 22:14:35.759735 19130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8218c1ef74471aa002c66d5b84eada" candidate_uuid: "0b91e34238f04b06baa97240e7b69754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d369bd03946448097f077439289bf82"
I20260811 22:14:35.759725 18982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8218c1ef74471aa002c66d5b84eada" candidate_uuid: "0b91e34238f04b06baa97240e7b69754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feef14da689c444b95d1fcc36d0e7284"
I20260811 22:14:35.759910 18982 raft_consensus.cc:3037] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:35.759954 19130 raft_consensus.cc:3037] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:35.761441 19130 raft_consensus.cc:2454] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b91e34238f04b06baa97240e7b69754 in term 1.
I20260811 22:14:35.762053 19205 leader_election.cc:302] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b91e34238f04b06baa97240e7b69754, 7d369bd03946448097f077439289bf82; no voters:
I20260811 22:14:35.762108 18982 raft_consensus.cc:2454] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b91e34238f04b06baa97240e7b69754 in term 1.
I20260811 22:14:35.762378 19356 raft_consensus.cc:2781] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:35.762699 19356 raft_consensus.cc:686] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [term 1 LEADER]: Becoming Leader. State: Replica: 0b91e34238f04b06baa97240e7b69754, State: Running, Role: LEADER
I20260811 22:14:35.763123 19356 consensus_queue.cc:237] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } }
I20260811 22:14:35.767769 18821 catalog_manager.cc:5495] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b91e34238f04b06baa97240e7b69754 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "0b91e34238f04b06baa97240e7b69754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91e34238f04b06baa97240e7b69754" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "feef14da689c444b95d1fcc36d0e7284" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d369bd03946448097f077439289bf82" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41527 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:35.786887 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.775436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.17.198.1:46237)
0811 22:14:35.775609 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.775614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.775826 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:14:35.775995 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.775998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.776051 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.776136 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.776983 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.777559 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.778560 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.778785 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.778905 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.782018 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.782035 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.782053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.782089 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.784392 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.785111 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.785118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.785120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.785232 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.785625 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.785628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.785630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.786001 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.786185 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.786512 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.786656 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:35.822857 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.811735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.17.198.1:46237)
0811 22:14:35.811922 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.811929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.812200 (+ 271us) server_negotiation.cc:407] Connection header received
0811 22:14:35.812405 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.812408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.812459 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.812557 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.813521 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.814100 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.815365 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.815619 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.815758 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.818875 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.818880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.818882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.818914 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.820678 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.821327 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.821332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.821334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.821438 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.821775 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.821779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.821781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.821989 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.822146 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.822515 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.822665 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:14:35.823876 19183 tablet.cc:2369] T bc8218c1ef74471aa002c66d5b84eada P 7d369bd03946448097f077439289bf82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:35.826248 19035 tablet.cc:2369] T bc8218c1ef74471aa002c66d5b84eada P feef14da689c444b95d1fcc36d0e7284: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[18774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:35.855948 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:35.845125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.17.198.1:46237)
0811 22:14:35.845309 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:35.845312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:35.845401 (+ 89us) server_negotiation.cc:407] Connection header received
0811 22:14:35.845535 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:35.845538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:35.845584 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:35.845664 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:35.846536 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.847049 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.848075 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:35.848339 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:35.848460 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:35.851614 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:35.851618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:35.851620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:35.851659 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:35.853600 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.854179 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.854183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.854184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.854302 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:35.854827 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:35.854830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:35.854831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:35.855066 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:35.855182 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:35.855550 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 22:14:35.855720 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 22:14:35.857656 18938 tablet_service.cc:2908] Rejecting scan request for tablet bc8218c1ef74471aa002c66d5b84eada: Uninitialized: clean time has not yet been initialized
I20260811 22:14:35.860334 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18891
I20260811 22:14:35.868584 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19039
W20260811 22:14:35.871423 19330 tablet.cc:2369] T bc8218c1ef74471aa002c66d5b84eada P 0b91e34238f04b06baa97240e7b69754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:35.877465 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19188
I20260811 22:14:35.885416 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 18789
2026-08-11T22:14:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2720 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[19376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[19376](info): set up 2 sockets
Aug 11 22:14:35 dist-test-slave-77v9 krb5kdc[19376](info): commencing operation
krb5kdc: starting...
W20260811 22:14:37.055094 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:37.201264 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:34295
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:34295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:37.323531 19392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:37.323951 19392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:37.324048 19392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:37.328961 19392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:37.329090 19392 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:14:37.329125 19392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:37.329175 19392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:37.329239 19392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:37.329283 19392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:37.335438 19392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:34295
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:34295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:37.336689 19392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:37.337817 19392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:14:37.347774 19392 server_base.cc:1034] running on GCE node
W20260811 22:14:37.347707 19398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:37.347707 19397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:37.347707 19400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:37.348630 19392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:37.349897 19392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:37.351123 19392 hybrid_clock.cc:648] HybridClock initialized: now 1786486477351083 us; error 56 us; skew 500 ppm
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:37.354372 19392 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:37.355407 19392 webserver.cc:460] Webserver started at http://127.17.198.62:38909/ using document root <none> and password file <none>
I20260811 22:14:37.355993 19392 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:37.356060 19392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:37.356248 19392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:37.359239 19392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "155b23b138374cc1973501d1d28796c3"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "6023c153834f7d949868fe159106a62a"
server_key_iv: "763df67a65d85291d33339c4db4fe264"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.360414 19392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "155b23b138374cc1973501d1d28796c3"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "6023c153834f7d949868fe159106a62a"
server_key_iv: "763df67a65d85291d33339c4db4fe264"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.365267 19392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:37.368014 19409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:37.369660 19392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:37.369823 19392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "155b23b138374cc1973501d1d28796c3"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "6023c153834f7d949868fe159106a62a"
server_key_iv: "763df67a65d85291d33339c4db4fe264"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.369987 19392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:37.389664 19392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:37.393429 19392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:37.393663 19392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:37.403007 19392 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:34295
I20260811 22:14:37.403043 19483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:34295 every 8 connection(s)
I20260811 22:14:37.404065 19392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:37.407392 19484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:37.410413 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19392
I20260811 22:14:37.410559 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:37.410920 18200 external_mini_cluster.cc:1428] Setting key 4a09eb79a96557beb242d43fbb2c8c00
I20260811 22:14:37.414620 19484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Bootstrap starting.
I20260811 22:14:37.417295 19484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:37.418192 19484 log.cc:824] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:37.420490 19484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: No bootstrap required, opened a new log
I20260811 22:14:37.424177 19484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } }
I20260811 22:14:37.424484 19484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:37.424568 19484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155b23b138374cc1973501d1d28796c3, State: Initialized, Role: FOLLOWER
I20260811 22:14:37.425148 19484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } }
I20260811 22:14:37.425320 19484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:37.425413 19484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:37.425556 19484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:37.426069 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:37.412547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.17.198.62:34295)
0811 22:14:37.413154 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:37.413171 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:37.413209 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:14:37.413886 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:37.413904 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:37.414250 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:37.414609 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:37.415475 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.416504 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.417508 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.418007 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.418193 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:37.420975 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:37.420980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:37.420996 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:37.421025 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:37.422734 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.423236 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.423242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.423246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.423374 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.423694 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.423698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.423699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.424130 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:37.424359 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:37.424701 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 22:14:37.425077 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":195,"threads_started":1}
I20260811 22:14:37.426960 19484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } }
I20260811 22:14:37.427407 19484 leader_election.cc:302] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 155b23b138374cc1973501d1d28796c3; no voters:
I20260811 22:14:37.427863 19484 leader_election.cc:288] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:37.428335 19494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:37.428630 19494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [term 1 LEADER]: Becoming Leader. State: Replica: 155b23b138374cc1973501d1d28796c3, State: Running, Role: LEADER
I20260811 22:14:37.429418 19484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:37.429867 19494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } }
I20260811 22:14:37.431309 19493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "155b23b138374cc1973501d1d28796c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } } }
I20260811 22:14:37.431600 19493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:37.431316 19494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155b23b138374cc1973501d1d28796c3. Latest consensus state: current_term: 1 leader_uuid: "155b23b138374cc1973501d1d28796c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155b23b138374cc1973501d1d28796c3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 34295 } } }
I20260811 22:14:37.432556 19494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3 [sys.catalog]: This master's current role is: LEADER
W20260811 22:14:37.436309 19507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:37.436491 19507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:37.436616 19509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:37.437657 19509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:37.444337 19509 catalog_manager.cc:1347] Generated new cluster ID: 436a3940e60e4430a72d43ab61088ddc
I20260811 22:14:37.444443 19509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:37.464416 19509 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:37.465842 19509 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:37.472764 19509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Generated new TSK 0
I20260811 22:14:37.473774 19509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:37.516868 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:34295
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:37.651961 19515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:37.652361 19515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:37.652458 19515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:37.657617 19515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:37.657735 19515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:37.657823 19515 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:37.657922 19515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:37.664422 19515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:34295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:37.665762 19515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:37.666977 19515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:14:37.678517 19515 server_base.cc:1034] running on GCE node
W20260811 22:14:37.678431 19521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:37.678435 19520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:37.678426 19524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:37.679332 19515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:37.680088 19515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:37.681272 19515 hybrid_clock.cc:648] HybridClock initialized: now 1786486477681238 us; error 50 us; skew 500 ppm
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:37.684698 19515 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:37.685938 19515 webserver.cc:460] Webserver started at http://127.17.198.1:33749/ using document root <none> and password file <none>
I20260811 22:14:37.686637 19515 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:37.686753 19515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:37.687034 19515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:37.689528 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "5e347aecfaf84a6a8d11e672ee94572a"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "08fc08abe628a13ae9bbf381c1cdbfec"
server_key_iv: "2a4b7db74d88de160cf1bc2a2ad84963"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.690235 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "5e347aecfaf84a6a8d11e672ee94572a"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "08fc08abe628a13ae9bbf381c1cdbfec"
server_key_iv: "2a4b7db74d88de160cf1bc2a2ad84963"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.694772 19515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:37.697909 19533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:37.699388 19515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:14:37.699581 19515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "5e347aecfaf84a6a8d11e672ee94572a"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "08fc08abe628a13ae9bbf381c1cdbfec"
server_key_iv: "2a4b7db74d88de160cf1bc2a2ad84963"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.699733 19515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:37.737453 19515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:37.741381 19515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:37.741678 19515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:37.743409 19515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:37.743507 19515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:37.743574 19515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:37.743613 19515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:37.756738 19515 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:43589
I20260811 22:14:37.756897 19654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:43589 every 8 connection(s)
I20260811 22:14:37.758081 19515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:37.759959 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:37.744478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:36233 (local address 127.17.198.62:34295)
0811 22:14:37.744619 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:37.744623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:37.745948 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 22:14:37.747124 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:37.747131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:37.747203 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:37.747268 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:37.748926 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.749491 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.750928 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.751346 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.751470 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:37.755462 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:37.755466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:37.755468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:37.755508 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:37.757467 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.758046 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.758053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.758056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.758169 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.758627 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.758630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.758632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.758880 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:37.759009 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:37.759681 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 22:14:37.759819 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:14:37.760948 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:37.745188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.1:36233)
0811 22:14:37.745766 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:37.745814 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:37.746841 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:37.747789 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:37.747796 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:37.748172 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:37.748726 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:37.748746 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.749634 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.749637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:37.750795 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:37.750804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.751521 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.751526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:37.751766 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:37.752704 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:37.752742 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:37.755223 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:37.757613 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:37.757618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:37.757630 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:37.757899 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:37.758304 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:37.758307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:37.758308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:37.758496 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:37.759054 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:37.759061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:37.759414 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 22:14:37.759606 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:14:37.759950 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":169,"threads_started":1}
I20260811 22:14:37.762421 19655 heartbeater.cc:347] Connected to a master server at 127.17.198.62:34295
I20260811 22:14:37.762646 19655 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:37.763226 19655 heartbeater.cc:511] Master 127.17.198.62:34295 requested a full tablet report, sending...
I20260811 22:14:37.765056 19427 ts_manager.cc:194] Registered new tserver with Master: 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1:43589)
I20260811 22:14:37.766366 19427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:36233
I20260811 22:14:37.766798 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19515
I20260811 22:14:37.766917 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:37.767233 18200 external_mini_cluster.cc:1428] Setting key 22d62281cc028b10c391d9abebe795c6
I20260811 22:14:37.775898 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:37.765328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:51365 (local address 127.17.198.62:34295)
0811 22:14:37.765475 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:37.765480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:37.766851 (+ 1371us) server_negotiation.cc:407] Connection header received
0811 22:14:37.767055 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:37.767060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:37.767116 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:37.767202 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:37.767924 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.768500 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.769740 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.770141 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.770272 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:37.771828 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:37.771831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:37.771833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:37.771858 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:37.773540 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.774207 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.774212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.774213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.774343 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:37.774747 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:37.774750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:37.774751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:37.775096 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:37.775183 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:37.775645 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 22:14:37.775766 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:14:37.776108 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:37.765254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.1:51365)
0811 22:14:37.766727 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:37.766745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:37.766878 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:37.767345 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:37.767349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:37.767653 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:37.767790 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:37.767798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.768651 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.768655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:37.769599 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:37.769609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:37.770288 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:37.770292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:37.770399 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:37.771033 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:37.771054 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:37.771646 (+ 592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:37.773688 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:37.773696 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:37.773702 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:37.774051 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:37.774482 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:37.774488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:37.774493 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:37.774613 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:37.775245 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:37.775252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:37.775420 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 22:14:37.775580 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:14:37.775808 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:37.805442 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:34295
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:37.928256 19667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:37.928582 19667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:37.928630 19667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:37.933388 19667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:37.933470 19667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:37.933514 19667 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:37.933549 19667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:37.939478 19667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:34295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:37.940637 19667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:37.941845 19667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:37.952723 19672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:37.953043 19667 server_base.cc:1034] running on GCE node
W20260811 22:14:37.952728 19676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:37.952788 19674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:37.953716 19667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:37.954484 19667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:37.955683 19667 hybrid_clock.cc:648] HybridClock initialized: now 1786486477955643 us; error 58 us; skew 500 ppm
Aug 11 22:14:37 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:37.959022 19667 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:37.960026 19667 webserver.cc:460] Webserver started at http://127.17.198.2:37787/ using document root <none> and password file <none>
I20260811 22:14:37.960625 19667 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:37.960691 19667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:37.960960 19667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:37.963469 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "2dcdee7ece734ab285a66baa05bf6591"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "5e80e95599aa6b7949d77bd066b4e810"
server_key_iv: "8361005693079ccd3c26ea26f23d2afb"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.964049 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "2dcdee7ece734ab285a66baa05bf6591"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "5e80e95599aa6b7949d77bd066b4e810"
server_key_iv: "8361005693079ccd3c26ea26f23d2afb"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.968281 19667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:37.971352 19684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:37.973254 19667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:37.973436 19667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "2dcdee7ece734ab285a66baa05bf6591"
format_stamp: "Formatted at 2026-08-11 22:14:37 on dist-test-slave-77v9"
server_key: "5e80e95599aa6b7949d77bd066b4e810"
server_key_iv: "8361005693079ccd3c26ea26f23d2afb"
server_key_version: "encryptionkey@0"
I20260811 22:14:37.973611 19667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:38.030632 19667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:38.034592 19667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:38.034914 19667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:38.036474 19667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:38.036557 19667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:38.036628 19667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:38.036679 19667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:38.050751 19667 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43197
I20260811 22:14:38.051548 19818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43197 every 8 connection(s)
I20260811 22:14:38.052165 19667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:38.053882 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.037478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:45473 (local address 127.17.198.62:34295)
0811 22:14:38.037670 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.037676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.039053 (+ 1377us) server_negotiation.cc:407] Connection header received
0811 22:14:38.040440 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.040445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.040508 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.040620 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.042388 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.042934 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.044247 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.044604 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.044727 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.049164 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.049169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.049172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.049213 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.051178 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.051973 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.051977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.051979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.052086 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.052544 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.052550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.052552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.052876 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.052992 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.053625 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.053755 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:14:38.054818 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.038386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.2:45473)
0811 22:14:38.038870 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.038926 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.040163 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.040860 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.040869 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.041544 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:38.042209 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.042227 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.043076 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.043080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.044100 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.044110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.044832 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.044839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.045077 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.045911 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:38.045955 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:38.048898 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:38.051346 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.051353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.051370 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.051833 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.052206 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.052211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.052213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.052407 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.053053 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:38.053060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:38.053387 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.053567 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.053854 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":133,"threads_started":1}
I20260811 22:14:38.055526 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19667
I20260811 22:14:38.055665 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:38.056021 18200 external_mini_cluster.cc:1428] Setting key 74aac37fb380415363fd51fa4c9ec23a
I20260811 22:14:38.056633 19819 heartbeater.cc:347] Connected to a master server at 127.17.198.62:34295
I20260811 22:14:38.056877 19819 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:38.057566 19819 heartbeater.cc:511] Master 127.17.198.62:34295 requested a full tablet report, sending...
I20260811 22:14:38.058863 19426 ts_manager.cc:194] Registered new tserver with Master: 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197)
I20260811 22:14:38.059754 19426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:45473
I20260811 22:14:38.069506 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.059304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.2:37167)
0811 22:14:38.059843 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.059861 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.059980 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.060558 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.060562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.060858 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:38.060999 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.061008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.062200 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.062204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.063301 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.063315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.064071 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.064074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.064159 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.064882 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:38.064899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:38.065299 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:38.067648 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.067652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.067653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.067899 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.068266 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.068269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.068270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.068360 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.068987 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:38.068989 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:38.069085 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.069174 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.069313 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":113,"threads_started":1}
I20260811 22:14:38.069610 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.059392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:37167 (local address 127.17.198.62:34295)
0811 22:14:38.059569 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.059578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.059973 (+ 395us) server_negotiation.cc:407] Connection header received
0811 22:14:38.060256 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.060259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.060318 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.060420 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.061149 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.061869 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.063477 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.063857 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.064010 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.065477 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.065484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.065489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.065531 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.067493 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.068031 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.068037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.068041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.068149 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.068481 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.068486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.068489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.068881 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.069011 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.069277 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.069422 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:38.094468 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:34295
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:38.218148 19829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:38.218518 19829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:38.218600 19829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:38.222982 19829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:38.223104 19829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:38.223184 19829 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:38.223273 19829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:38.229162 19829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:34295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:38.230491 19829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:38.231715 19829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:38.243136 19839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:38.243084 19837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:38.243069 19836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:38.243525 19829 server_base.cc:1034] running on GCE node
I20260811 22:14:38.244088 19829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:38.244877 19829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:38.246093 19829 hybrid_clock.cc:648] HybridClock initialized: now 1786486478246071 us; error 36 us; skew 500 ppm
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486478, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:38.249998 19829 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:38.251137 19829 webserver.cc:460] Webserver started at http://127.17.198.3:37259/ using document root <none> and password file <none>
I20260811 22:14:38.251811 19829 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:38.251922 19829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:38.252175 19829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:38.254653 19829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
format_stamp: "Formatted at 2026-08-11 22:14:38 on dist-test-slave-77v9"
server_key: "0c6e3252b09fc809d6b58c7662da94ae"
server_key_iv: "90134f4079586ca14dbe07986bc6a35f"
server_key_version: "encryptionkey@0"
I20260811 22:14:38.255359 19829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
format_stamp: "Formatted at 2026-08-11 22:14:38 on dist-test-slave-77v9"
server_key: "0c6e3252b09fc809d6b58c7662da94ae"
server_key_iv: "90134f4079586ca14dbe07986bc6a35f"
server_key_version: "encryptionkey@0"
I20260811 22:14:38.259928 19829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:14:38.263271 19846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:38.265002 19829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:38.265172 19829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
format_stamp: "Formatted at 2026-08-11 22:14:38 on dist-test-slave-77v9"
server_key: "0c6e3252b09fc809d6b58c7662da94ae"
server_key_iv: "90134f4079586ca14dbe07986bc6a35f"
server_key_version: "encryptionkey@0"
I20260811 22:14:38.265353 19829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:38.291507 19829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:38.295951 19829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:38.296271 19829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:38.298292 19829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:38.298358 19829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:38.298403 19829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:38.298578 19829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486478, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:38.320277 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.299669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:41755 (local address 127.17.198.62:34295)
0811 22:14:38.299890 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.299898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.301018 (+ 1120us) server_negotiation.cc:407] Connection header received
0811 22:14:38.302184 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.302188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.302298 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.302695 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.304456 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.305351 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.308220 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.308675 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.309014 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.313731 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.313735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.313738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.313772 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.316115 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.317345 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.317350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.317352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.317526 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.318242 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.318251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.318252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.318614 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.318835 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.319842 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.320032 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:14:38.321280 19829 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:39891
I20260811 22:14:38.321480 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.300088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.3:41755)
0811 22:14:38.300731 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.300867 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.301904 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.302951 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.302959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.303508 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:38.304245 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.304266 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.305594 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.305602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.307942 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.307976 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.309203 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.309206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.309403 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.310410 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:38.310431 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:38.313358 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:38.316336 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.316346 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.316369 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.317120 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.317747 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.317749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.317751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.318063 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.318904 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:38.318962 (+ 58us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:38.319439 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.319732 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.320213 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":139,"threads_started":1}
I20260811 22:14:38.322418 19975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:39891 every 8 connection(s)
I20260811 22:14:38.322642 19829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:38.324095 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 19829
I20260811 22:14:38.324170 19976 heartbeater.cc:347] Connected to a master server at 127.17.198.62:34295
I20260811 22:14:38.324247 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:38.324349 19976 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:38.324622 18200 external_mini_cluster.cc:1428] Setting key 264418789ab5e223fc9fa65c48f0be84
I20260811 22:14:38.325280 19976 heartbeater.cc:511] Master 127.17.198.62:34295 requested a full tablet report, sending...
I20260811 22:14:38.326399 19427 ts_manager.cc:194] Registered new tserver with Master: 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891)
I20260811 22:14:38.327267 19427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:41755
I20260811 22:14:38.328272 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:38.337579 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.326412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.17.198.3:50615)
0811 22:14:38.326970 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.326987 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.327099 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.327418 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.327421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.328154 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:38.328398 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.328407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.329652 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.329658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.330342 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.330352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.331274 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.331281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.331469 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.332540 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:38.332563 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:38.333157 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:38.335489 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.335494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.335496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.335811 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.336146 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.336152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.336157 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.336279 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.336683 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:38.336689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:38.336878 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.337009 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.337214 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":1,"thread_start_us":158,"threads_started":1}
I20260811 22:14:38.337618 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.326327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:50615 (local address 127.17.198.62:34295)
0811 22:14:38.326457 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.326462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.327102 (+ 640us) server_negotiation.cc:407] Connection header received
0811 22:14:38.327208 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.327211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.327277 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.327363 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.328518 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.329453 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.330480 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.331084 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.331267 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.333360 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.333363 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.333366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.333394 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.335307 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.335953 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.335956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.335958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.336043 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.336398 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.336400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.336401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.336589 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.336667 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.337074 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.337358 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:38.361512 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.351524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.17.198.62:34295)
0811 22:14:38.351678 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.351682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.351696 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:38.351744 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.351746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.351787 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.351864 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.352969 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.353519 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.354408 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.354644 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.354725 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.357694 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.357698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.357700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.357779 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.359606 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.360105 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.360108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.360110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.360207 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.360465 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.360467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.360468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.360860 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.361001 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.361257 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.361376 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:14:38.365278 19426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:38.368495 19426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:38.385579 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.378528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43589 (local address 127.0.0.1:50646)
0811 22:14:38.378945 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.379008 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.379186 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.380123 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.380133 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.380163 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:38.380376 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.380387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.382265 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.382268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.383725 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.383735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.384966 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.384969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.385097 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.385143 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.385260 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.385390 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
I20260811 22:14:38.385622 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.378687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.17.198.1:43589)
0811 22:14:38.378991 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.378999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.379328 (+ 329us) server_negotiation.cc:407] Connection header received
0811 22:14:38.379626 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.379634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.379911 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.380597 (+ 686us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:38.380759 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.382089 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.383875 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.384742 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.385034 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.385235 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.385308 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.385440 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260811 22:14:38.386158 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.379588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:39891 (local address 127.0.0.1:50428)
0811 22:14:38.379911 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.379928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.380064 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.381756 (+ 1692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.381762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.381784 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:38.381900 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.381906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.383279 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.383283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.384369 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.384379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.385716 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.385720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.385840 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.385878 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.385934 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.386019 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":143,"threads_started":1}
I20260811 22:14:38.386747 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.380072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43197 (local address 127.0.0.1:41346)
0811 22:14:38.380411 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.380424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.380607 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.381451 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.381454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.381472 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:38.381682 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.381689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.383528 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.383531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.384683 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.384694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.386266 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.386270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.386436 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.386463 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.386538 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.386617 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":175,"threads_started":1}
I20260811 22:14:38.387068 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.379787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.17.198.3:39891)
0811 22:14:38.381242 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.381286 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.381311 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:14:38.381388 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.381393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.381579 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.381768 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:38.382049 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.383134 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.384525 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.385567 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.386557 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.386746 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.386808 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.386913 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1342,"thread_start_us":275,"threads_started":1}
I20260811 22:14:38.387763 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.380642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.17.198.2:43197)
0811 22:14:38.381040 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.381049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.381068 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:14:38.381130 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.381133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.381281 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.381472 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:38.381841 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.383394 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.385034 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.386092 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.387362 (+ 1270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.387541 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.387586 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.387657 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":164,"threads_started":1}
I20260811 22:14:38.390062 19585 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d734df0524baf81ca8f905842a52e (DEFAULT_TABLE table=test-table [id=c71f9c999cba457a9501f775d4bdd38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.390285 19742 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d734df0524baf81ca8f905842a52e (DEFAULT_TABLE table=test-table [id=c71f9c999cba457a9501f775d4bdd38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.390446 19904 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d734df0524baf81ca8f905842a52e (DEFAULT_TABLE table=test-table [id=c71f9c999cba457a9501f775d4bdd38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.391505 19742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d734df0524baf81ca8f905842a52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.391554 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d734df0524baf81ca8f905842a52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.391637 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d734df0524baf81ca8f905842a52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.399859 20008 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Bootstrap starting.
I20260811 22:14:38.400978 20009 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Bootstrap starting.
I20260811 22:14:38.401579 20010 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Bootstrap starting.
I20260811 22:14:38.403364 20008 tablet_bootstrap.cc:654] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.403962 20009 tablet_bootstrap.cc:654] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.404335 20008 log.cc:824] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:38.404896 20009 log.cc:824] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:38.406291 20010 tablet_bootstrap.cc:654] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.407140 20010 log.cc:824] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:38.409492 20008 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: No bootstrap required, opened a new log
I20260811 22:14:38.409826 20008 ts_tablet_manager.cc:1397] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 22:14:38.410514 20009 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: No bootstrap required, opened a new log
I20260811 22:14:38.410538 20010 tablet_bootstrap.cc:492] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: No bootstrap required, opened a new log
I20260811 22:14:38.410707 20009 ts_tablet_manager.cc:1397] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 22:14:38.410742 20010 ts_tablet_manager.cc:1397] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 22:14:38.415562 20008 raft_consensus.cc:348] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.415562 20010 raft_consensus.cc:348] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.415930 20008 raft_consensus.cc:374] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.415930 20010 raft_consensus.cc:374] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.415741 20009 raft_consensus.cc:348] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.415993 20010 raft_consensus.cc:729] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfa7d9f28734dd5bae2abf7d329b3ea, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.415993 20008 raft_consensus.cc:729] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcdee7ece734ab285a66baa05bf6591, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.416030 20009 raft_consensus.cc:374] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.416078 20009 raft_consensus.cc:729] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e347aecfaf84a6a8d11e672ee94572a, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.416558 20009 consensus_queue.cc:260] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.416558 20010 consensus_queue.cc:260] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.416610 20008 consensus_queue.cc:260] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.417891 20008 ts_tablet_manager.cc:1428] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:14:38.417891 20010 ts_tablet_manager.cc:1428] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:14:38.418063 20009 ts_tablet_manager.cc:1428] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:14:38.418352 19976 heartbeater.cc:503] Master 127.17.198.62:34295 was elected leader, sending a full tablet report...
I20260811 22:14:38.418350 19819 heartbeater.cc:503] Master 127.17.198.62:34295 was elected leader, sending a full tablet report...
I20260811 22:14:38.418561 19655 heartbeater.cc:503] Master 127.17.198.62:34295 was elected leader, sending a full tablet report...
I20260811 22:14:38.467129 20017 raft_consensus.cc:482] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:38.467407 20017 raft_consensus.cc:504] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.468768 20017 leader_election.cc:288] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891), 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197)
I20260811 22:14:38.473982 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.469214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:53217 (local address 127.17.198.3:39891)
0811 22:14:38.469365 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.469369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.469528 (+ 159us) server_negotiation.cc:407] Connection header received
0811 22:14:38.469705 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.469708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.469787 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.469896 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:38.470221 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.471271 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.472543 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.473171 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.473283 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.473328 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.473652 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.473742 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:14:38.474033 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.469808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:36215 (local address 127.17.198.2:43197)
0811 22:14:38.469937 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.469992 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.470258 (+ 266us) server_negotiation.cc:407] Connection header received
0811 22:14:38.470462 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.470465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.470522 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.470609 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:38.470996 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.471583 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.472757 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.473379 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.473508 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.473580 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.473821 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.473904 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:14:38.474085 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.469586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43197 (local address 127.17.198.1:36215)
0811 22:14:38.470122 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.470134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.470277 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.470698 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.470701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.470721 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:38.470849 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.470857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.471719 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.471721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.472616 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.472626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.473531 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.473533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.473689 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.473709 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.473775 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.473837 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":5,"thread_start_us":119,"threads_started":1}
I20260811 22:14:38.474479 20018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.469059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:39891 (local address 127.17.198.1:53217)
0811 22:14:38.469411 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.469428 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.469569 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.469934 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.469937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.469955 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:38.470096 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.470102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.471392 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.471395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.472382 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.472393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.473307 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.473312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.473452 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.473488 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.474242 (+ 754us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.474343 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":25,"thread_start_us":167,"threads_started":1}
I20260811 22:14:38.474989 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d734df0524baf81ca8f905842a52e" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dcdee7ece734ab285a66baa05bf6591" is_pre_election: true
I20260811 22:14:38.475147 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d734df0524baf81ca8f905842a52e" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" is_pre_election: true
I20260811 22:14:38.475406 19928 raft_consensus.cc:2454] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 0.
I20260811 22:14:38.475407 19767 raft_consensus.cc:2454] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 0.
I20260811 22:14:38.476027 19537 leader_election.cc:302] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcdee7ece734ab285a66baa05bf6591, 5e347aecfaf84a6a8d11e672ee94572a; no voters:
I20260811 22:14:38.476328 20017 raft_consensus.cc:2781] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:38.476428 20017 raft_consensus.cc:482] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:38.476475 20017 raft_consensus.cc:3037] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.477748 20017 raft_consensus.cc:504] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.478222 20017 leader_election.cc:288] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 election: Requested vote from peers 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891), 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197)
I20260811 22:14:38.478819 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d734df0524baf81ca8f905842a52e" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
I20260811 22:14:38.478838 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d734df0524baf81ca8f905842a52e" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dcdee7ece734ab285a66baa05bf6591"
I20260811 22:14:38.478981 19928 raft_consensus.cc:3037] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.478981 19767 raft_consensus.cc:3037] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.480508 19928 raft_consensus.cc:2454] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 1.
I20260811 22:14:38.480508 19767 raft_consensus.cc:2454] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 1.
I20260811 22:14:38.480909 19537 leader_election.cc:302] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcdee7ece734ab285a66baa05bf6591, 5e347aecfaf84a6a8d11e672ee94572a; no voters:
I20260811 22:14:38.481178 20017 raft_consensus.cc:2781] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:38.481503 20017 raft_consensus.cc:686] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 1 LEADER]: Becoming Leader. State: Replica: 5e347aecfaf84a6a8d11e672ee94572a, State: Running, Role: LEADER
I20260811 22:14:38.481886 20017 consensus_queue.cc:237] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } }
I20260811 22:14:38.485513 19425 catalog_manager.cc:5495] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1). New cstate: current_term: 1 leader_uuid: "5e347aecfaf84a6a8d11e672ee94572a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } health_report { overall_health: UNKNOWN } } }
W20260811 22:14:38.509590 19656 tablet.cc:2369] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:38.520224 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.516130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.17.198.1:43589)
0811 22:14:38.516285 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.516289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.516306 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:38.516455 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.516458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.516519 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.516636 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:38.517023 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.517588 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.518486 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.518729 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.518823 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.519178 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:38.519660 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:38.519758 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.520029 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.520097 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:14:38.527618 19767 raft_consensus.cc:1261] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Refusing update from remote peer 5e347aecfaf84a6a8d11e672ee94572a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:38.527968 19928 raft_consensus.cc:1261] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Refusing update from remote peer 5e347aecfaf84a6a8d11e672ee94572a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:38.528563 20020 consensus_queue.cc:1035] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:38.528729 20017 consensus_queue.cc:1035] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:38.539554 20025 mvcc.cc:205] Tried to move back new op lower bound from 7317448616039510016 to 7317448615864561664. Current Snapshot: MvccSnapshot[applied={T|T < 7317448616039510016 or (T in {7317448616039510016})}]
I20260811 22:14:38.552562 19427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:38.552803 19427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:38.556231 19427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Sending DeleteTablet for 3 replicas of tablet 7f0d734df0524baf81ca8f905842a52e
I20260811 22:14:38.557287 19585 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d734df0524baf81ca8f905842a52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:38 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50646
I20260811 22:14:38.557489 19904 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d734df0524baf81ca8f905842a52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:38 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50428
I20260811 22:14:38.557621 19742 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d734df0524baf81ca8f905842a52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:38 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:41346
I20260811 22:14:38.558161 20033 tablet_replica.cc:330] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: stopping tablet replica
I20260811 22:14:38.558360 20036 tablet_replica.cc:330] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: stopping tablet replica
I20260811 22:14:38.558668 20033 raft_consensus.cc:2229] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:38.558715 20036 raft_consensus.cc:2229] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:38.559140 20036 raft_consensus.cc:2258] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:38.559160 20033 raft_consensus.cc:2258] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:38.559195 19427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:38.559725 19427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:38.561025 20036 ts_tablet_manager.cc:1905] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:38.563905 20033 ts_tablet_manager.cc:1905] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:38.564361 20036 ts_tablet_manager.cc:1918] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:38.564567 20036 log.cc:1196] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/7f0d734df0524baf81ca8f905842a52e
I20260811 22:14:38.565090 20036 ts_tablet_manager.cc:1939] T 7f0d734df0524baf81ca8f905842a52e P 2dcdee7ece734ab285a66baa05bf6591: Deleting consensus metadata
I20260811 22:14:38.566098 19742 tablet_service.cc:1467] Processing CreateTablet for tablet a85c91f728b74208bfbe043ddac49ac7 (DEFAULT_TABLE table=test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.566397 19742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85c91f728b74208bfbe043ddac49ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.566658 19413 catalog_manager.cc:4841] TS 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197): tablet 7f0d734df0524baf81ca8f905842a52e (table test-table [id=c71f9c999cba457a9501f775d4bdd38c]) successfully deleted
I20260811 22:14:38.567682 19585 tablet_service.cc:1467] Processing CreateTablet for tablet a85c91f728b74208bfbe043ddac49ac7 (DEFAULT_TABLE table=test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.568022 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85c91f728b74208bfbe043ddac49ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.568527 20035 tablet_replica.cc:330] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: stopping tablet replica
I20260811 22:14:38.568995 20035 raft_consensus.cc:2229] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:38.569375 20035 raft_consensus.cc:2258] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:38.569399 20008 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Bootstrap starting.
I20260811 22:14:38.571331 20008 tablet_bootstrap.cc:654] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.571487 20009 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Bootstrap starting.
I20260811 22:14:38.571499 20033 ts_tablet_manager.cc:1918] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:38.571702 20033 log.cc:1196] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/7f0d734df0524baf81ca8f905842a52e
I20260811 22:14:38.572043 20033 ts_tablet_manager.cc:1939] T 7f0d734df0524baf81ca8f905842a52e P 5e347aecfaf84a6a8d11e672ee94572a: Deleting consensus metadata
I20260811 22:14:38.572964 20009 tablet_bootstrap.cc:654] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.574122 19904 tablet_service.cc:1467] Processing CreateTablet for tablet a85c91f728b74208bfbe043ddac49ac7 (DEFAULT_TABLE table=test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:38.574460 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85c91f728b74208bfbe043ddac49ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.574450 20008 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: No bootstrap required, opened a new log
I20260811 22:14:38.574548 20008 ts_tablet_manager.cc:1397] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:14:38.575139 20008 raft_consensus.cc:348] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.575261 20008 raft_consensus.cc:374] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.575296 20008 raft_consensus.cc:729] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcdee7ece734ab285a66baa05bf6591, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.575430 20008 consensus_queue.cc:260] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.575735 20008 ts_tablet_manager.cc:1428] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:14:38.579133 20010 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Bootstrap starting.
I20260811 22:14:38.579337 20035 ts_tablet_manager.cc:1905] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:38.579967 20009 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: No bootstrap required, opened a new log
I20260811 22:14:38.580071 20009 ts_tablet_manager.cc:1397] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260811 22:14:38.580461 20010 tablet_bootstrap.cc:654] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.580721 20009 raft_consensus.cc:348] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.580926 20009 raft_consensus.cc:374] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.580971 20009 raft_consensus.cc:729] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e347aecfaf84a6a8d11e672ee94572a, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.581141 20009 consensus_queue.cc:260] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.581466 20009 ts_tablet_manager.cc:1428] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:14:38.582543 20010 tablet_bootstrap.cc:492] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: No bootstrap required, opened a new log
I20260811 22:14:38.582634 20010 ts_tablet_manager.cc:1397] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:14:38.583096 20010 raft_consensus.cc:348] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.583217 20010 raft_consensus.cc:374] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.583256 20010 raft_consensus.cc:729] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfa7d9f28734dd5bae2abf7d329b3ea, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.583489 20010 consensus_queue.cc:260] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.583565 19413 catalog_manager.cc:4841] TS 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1:43589): tablet 7f0d734df0524baf81ca8f905842a52e (table test-table [id=c71f9c999cba457a9501f775d4bdd38c]) successfully deleted
I20260811 22:14:38.584384 20010 ts_tablet_manager.cc:1428] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:14:38.584609 20035 ts_tablet_manager.cc:1918] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:38.584779 20035 log.cc:1196] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/7f0d734df0524baf81ca8f905842a52e
I20260811 22:14:38.585242 20035 ts_tablet_manager.cc:1939] T 7f0d734df0524baf81ca8f905842a52e P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting consensus metadata
I20260811 22:14:38.586658 19413 catalog_manager.cc:4841] TS 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891): tablet 7f0d734df0524baf81ca8f905842a52e (table test-table [id=c71f9c999cba457a9501f775d4bdd38c]) successfully deleted
W20260811 22:14:38.804745 19820 tablet.cc:2369] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:38.827162 19978 tablet.cc:2369] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:38.876070 20020 raft_consensus.cc:482] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:38.876227 20020 raft_consensus.cc:504] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.876698 20020 leader_election.cc:288] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197), 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891)
I20260811 22:14:38.877409 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85c91f728b74208bfbe043ddac49ac7" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dcdee7ece734ab285a66baa05bf6591" is_pre_election: true
I20260811 22:14:38.877409 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85c91f728b74208bfbe043ddac49ac7" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" is_pre_election: true
I20260811 22:14:38.877583 19767 raft_consensus.cc:2454] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 0.
I20260811 22:14:38.877583 19928 raft_consensus.cc:2454] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 0.
I20260811 22:14:38.878014 19537 leader_election.cc:302] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcdee7ece734ab285a66baa05bf6591, 5e347aecfaf84a6a8d11e672ee94572a; no voters:
I20260811 22:14:38.878213 20020 raft_consensus.cc:2781] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:38.878278 20020 raft_consensus.cc:482] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:38.878340 20020 raft_consensus.cc:3037] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.879508 20020 raft_consensus.cc:504] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.879881 20020 leader_election.cc:288] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 election: Requested vote from peers 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197), 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891)
I20260811 22:14:38.880311 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85c91f728b74208bfbe043ddac49ac7" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dcdee7ece734ab285a66baa05bf6591"
I20260811 22:14:38.880337 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85c91f728b74208bfbe043ddac49ac7" candidate_uuid: "5e347aecfaf84a6a8d11e672ee94572a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
I20260811 22:14:38.880473 19928 raft_consensus.cc:3037] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.880474 19767 raft_consensus.cc:3037] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:38.881708 19767 raft_consensus.cc:2454] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 1.
I20260811 22:14:38.881708 19928 raft_consensus.cc:2454] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e347aecfaf84a6a8d11e672ee94572a in term 1.
I20260811 22:14:38.882088 19537 leader_election.cc:302] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfa7d9f28734dd5bae2abf7d329b3ea, 5e347aecfaf84a6a8d11e672ee94572a; no voters:
I20260811 22:14:38.882282 20020 raft_consensus.cc:2781] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:38.882386 20020 raft_consensus.cc:686] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 1 LEADER]: Becoming Leader. State: Replica: 5e347aecfaf84a6a8d11e672ee94572a, State: Running, Role: LEADER
I20260811 22:14:38.882546 20020 consensus_queue.cc:237] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.884657 19425 catalog_manager.cc:5495] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1). New cstate: current_term: 1 leader_uuid: "5e347aecfaf84a6a8d11e672ee94572a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:14:38 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:38.980608 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.970987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:34295 (local address 127.0.0.1:38406)
0811 22:14:38.971352 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:38.971368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:38.971544 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:38.971820 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:38.971822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:38.972262 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:38.972364 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.972373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.973236 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.973241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.973868 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:38.973877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.974424 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.974432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:38.974551 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.975253 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:38.975265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:38.976785 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:38.978834 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.978845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.978851 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.979106 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.979446 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:38.979449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:38.979450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:38.979531 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:38.979957 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:38.979964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:38.980111 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 22:14:38.980232 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:14:38.980410 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
I20260811 22:14:38.980639 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:38.971067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.17.198.62:34295)
0811 22:14:38.971339 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:38.971349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:38.971476 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:14:38.971650 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:38.971653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:38.971701 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:38.971798 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:38.972499 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.973121 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.973990 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:38.974241 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:38.974364 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:38.977022 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:38.977025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:38.977027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:38.977051 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:38.978721 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.979199 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.979201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.979203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.979330 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:38.979615 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:38.979617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:38.979618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:38.979819 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:38.979940 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:38.980284 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 22:14:38.980499 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":226,"thread_start_us":142,"threads_started":1}
I20260811 22:14:38.985685 19425 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:38406:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:14:38.986656 19425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:38.992406 19585 tablet_service.cc:1467] Processing CreateTablet for tablet 9ce0cf0018b545728d36d441dbd0776c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b4d9271ac684c578d73ccda14783c93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:38.992406 19742 tablet_service.cc:1467] Processing CreateTablet for tablet 9ce0cf0018b545728d36d441dbd0776c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b4d9271ac684c578d73ccda14783c93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:38.992949 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ce0cf0018b545728d36d441dbd0776c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.992954 19742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ce0cf0018b545728d36d441dbd0776c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.995862 20009 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: Bootstrap starting.
I20260811 22:14:38.995908 20008 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: Bootstrap starting.
I20260811 22:14:38.996220 19904 tablet_service.cc:1467] Processing CreateTablet for tablet 9ce0cf0018b545728d36d441dbd0776c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b4d9271ac684c578d73ccda14783c93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:38.996549 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ce0cf0018b545728d36d441dbd0776c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:38.996941 20009 tablet_bootstrap.cc:654] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.996942 20008 tablet_bootstrap.cc:654] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:38.998374 20008 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: No bootstrap required, opened a new log
I20260811 22:14:38.998484 20008 ts_tablet_manager.cc:1397] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:14:38.998790 20009 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: No bootstrap required, opened a new log
I20260811 22:14:38.998924 20009 ts_tablet_manager.cc:1397] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:14:38.998978 20008 raft_consensus.cc:348] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.999112 20008 raft_consensus.cc:374] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.999143 20008 raft_consensus.cc:729] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcdee7ece734ab285a66baa05bf6591, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.999279 20008 consensus_queue.cc:260] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.999459 20009 raft_consensus.cc:348] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:38.999570 20008 ts_tablet_manager.cc:1428] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:14:38.999621 20009 raft_consensus.cc:374] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:38.999650 20010 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: Bootstrap starting.
I20260811 22:14:38.999670 20009 raft_consensus.cc:729] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e347aecfaf84a6a8d11e672ee94572a, State: Initialized, Role: FOLLOWER
I20260811 22:14:38.999626 20016 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:38.999785 20016 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:38.999820 20009 consensus_queue.cc:260] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.000129 20020 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.000286 20020 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.000593 20009 ts_tablet_manager.cc:1428] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:14:39.000758 20010 tablet_bootstrap.cc:654] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:39.002689 20010 tablet_bootstrap.cc:492] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: No bootstrap required, opened a new log
I20260811 22:14:39.002800 20010 ts_tablet_manager.cc:1397] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:39.003298 20010 raft_consensus.cc:348] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.003450 20010 raft_consensus.cc:374] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:39.003492 20010 raft_consensus.cc:729] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfa7d9f28734dd5bae2abf7d329b3ea, State: Initialized, Role: FOLLOWER
I20260811 22:14:39.003629 20010 consensus_queue.cc:260] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.003934 20010 ts_tablet_manager.cc:1428] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:14:39.003990 20014 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.004173 20014 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.016784 20016 raft_consensus.cc:482] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:39.017035 20016 raft_consensus.cc:504] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.018193 20016 leader_election.cc:288] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1:43589), 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891)
I20260811 22:14:39.022840 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.018517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:41641 (local address 127.17.198.1:43589)
0811 22:14:39.018707 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.018714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.018900 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:14:39.019178 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.019181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.019235 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.019316 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:39.019667 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.020306 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.021735 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.022153 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.022253 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.022296 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.022572 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.022673 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:14:39.022868 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.018428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43589 (local address 127.17.198.2:41641)
0811 22:14:39.018705 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:39.018721 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:39.018970 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:39.019376 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:39.019379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:39.019399 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:39.019538 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.019545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.020448 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.020453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.021591 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.021602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.022317 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.022325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.022423 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.022448 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:14:39.022538 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:14:39.022644 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":42,"thread_start_us":75,"threads_started":1}
I20260811 22:14:39.022850 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.018832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:39891 (local address 127.17.198.2:35661)
0811 22:14:39.019092 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:39.019126 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:39.019216 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:39.019541 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:39.019543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:39.019557 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:39.019701 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.019707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.020576 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.020580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.021556 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.021567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.022389 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.022392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.022491 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.022542 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:14:39.022625 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:14:39.022690 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":35,"thread_start_us":85,"threads_started":1}
I20260811 22:14:39.022984 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.018960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:35661 (local address 127.17.198.3:39891)
0811 22:14:39.019274 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.019278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.019312 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:14:39.019372 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.019376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.019426 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.019523 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:39.019833 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.020445 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.021700 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.022253 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.022394 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.022449 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.022663 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.022796 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":119,"threads_started":1}
I20260811 22:14:39.023659 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ce0cf0018b545728d36d441dbd0776c" candidate_uuid: "2dcdee7ece734ab285a66baa05bf6591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" is_pre_election: true
I20260811 22:14:39.023891 19928 raft_consensus.cc:2454] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dcdee7ece734ab285a66baa05bf6591 in term 0.
I20260811 22:14:39.023947 19607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ce0cf0018b545728d36d441dbd0776c" candidate_uuid: "2dcdee7ece734ab285a66baa05bf6591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e347aecfaf84a6a8d11e672ee94572a" is_pre_election: true
I20260811 22:14:39.024206 19607 raft_consensus.cc:2454] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dcdee7ece734ab285a66baa05bf6591 in term 0.
I20260811 22:14:39.024350 19688 leader_election.cc:302] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcdee7ece734ab285a66baa05bf6591, 5cfa7d9f28734dd5bae2abf7d329b3ea; no voters:
I20260811 22:14:39.024722 20016 raft_consensus.cc:2781] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:39.024784 20016 raft_consensus.cc:482] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:39.024860 20016 raft_consensus.cc:3037] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:39.025969 20016 raft_consensus.cc:504] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.026373 20016 leader_election.cc:288] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [CANDIDATE]: Term 1 election: Requested vote from peers 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1:43589), 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891)
I20260811 22:14:39.027011 19928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ce0cf0018b545728d36d441dbd0776c" candidate_uuid: "2dcdee7ece734ab285a66baa05bf6591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea"
I20260811 22:14:39.027011 19607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ce0cf0018b545728d36d441dbd0776c" candidate_uuid: "2dcdee7ece734ab285a66baa05bf6591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e347aecfaf84a6a8d11e672ee94572a"
I20260811 22:14:39.027169 19928 raft_consensus.cc:3037] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:39.027169 19607 raft_consensus.cc:3037] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:39.028465 19607 raft_consensus.cc:2454] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dcdee7ece734ab285a66baa05bf6591 in term 1.
I20260811 22:14:39.028481 19928 raft_consensus.cc:2454] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dcdee7ece734ab285a66baa05bf6591 in term 1.
I20260811 22:14:39.028980 19688 leader_election.cc:302] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcdee7ece734ab285a66baa05bf6591, 5cfa7d9f28734dd5bae2abf7d329b3ea; no voters:
I20260811 22:14:39.029225 20016 raft_consensus.cc:2781] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:39.029489 20016 raft_consensus.cc:686] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [term 1 LEADER]: Becoming Leader. State: Replica: 2dcdee7ece734ab285a66baa05bf6591, State: Running, Role: LEADER
I20260811 22:14:39.029825 20016 consensus_queue.cc:237] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } }
I20260811 22:14:39.031369 20057 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: TxnStatusTablet state changed. Reason: New leader 2dcdee7ece734ab285a66baa05bf6591. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.031543 20057 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:39.032114 20059 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:14:39.032405 20059 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:14:39.032764 19427 catalog_manager.cc:5495] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } health_report { overall_health: UNKNOWN } } }
I20260811 22:14:39.056150 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.050550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43197 (local address 127.0.0.1:60018)
0811 22:14:39.050709 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:39.050726 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:39.050847 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:39.051661 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:39.051665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:39.051676 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:39.051805 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.051812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.053092 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.053096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.054087 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.054099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.054747 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.054751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.054852 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.054929 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.055776 (+ 847us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.055795 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:14:39.055873 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:14:39.055997 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 22:14:39.056175 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.050665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.17.198.2:43197)
0811 22:14:39.051324 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.051329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.051346 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:39.051429 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.051432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.051496 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.051614 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:39.051957 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.052955 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.054233 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.054612 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.054721 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.055113 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.055671 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.055783 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.055903 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.055971 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":77,"threads_started":1}
I20260811 22:14:39.060505 19928 raft_consensus.cc:1261] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Refusing update from remote peer 2dcdee7ece734ab285a66baa05bf6591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:39.060628 19607 raft_consensus.cc:1261] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Refusing update from remote peer 2dcdee7ece734ab285a66baa05bf6591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:39.061488 20016 consensus_queue.cc:1035] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:39.061730 20057 consensus_queue.cc:1035] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:39.063748 20014 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: TxnStatusTablet state changed. Reason: New leader 2dcdee7ece734ab285a66baa05bf6591. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.063910 20014 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.063977 20020 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: TxnStatusTablet state changed. Reason: New leader 2dcdee7ece734ab285a66baa05bf6591. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.064112 20020 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.067440 20057 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.067621 20057 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:39.068236 20057 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.068399 20057 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 2dcdee7ece734ab285a66baa05bf6591: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:39.068246 20064 mvcc.cc:205] Tried to move back new op lower bound from 7317448618225283072 to 7317448618109145088. Current Snapshot: MvccSnapshot[applied={T|T < 7317448618225283072}]
I20260811 22:14:39.069621 20014 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.069774 20014 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5cfa7d9f28734dd5bae2abf7d329b3ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.070674 20020 tablet_replica.cc:439] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcdee7ece734ab285a66baa05bf6591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e347aecfaf84a6a8d11e672ee94572a" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43589 } } peers { permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 } } peers { permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 } } }
I20260811 22:14:39.070837 20020 tablet_replica.cc:442] T 9ce0cf0018b545728d36d441dbd0776c P 5e347aecfaf84a6a8d11e672ee94572a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:39.074368 20066 mvcc.cc:205] Tried to move back new op lower bound from 7317448618225283072 to 7317448618109145088. Current Snapshot: MvccSnapshot[applied={T|T < 7317448618225283072}]
I20260811 22:14:39.091444 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.087660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43197 (local address 127.17.198.1:38003)
0811 22:14:39.088030 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:39.088045 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:39.088198 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:39.088490 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:39.088493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:39.088501 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:39.088624 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.088629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.089470 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.089476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.090168 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.090178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.090658 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.090662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.090755 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.090810 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.091140 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.091150 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:14:39.091215 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:14:39.091309 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260811 22:14:39.091444 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.087800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38003 (local address 127.17.198.2:43197)
0811 22:14:39.087943 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.087948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.088128 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:14:39.088312 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.088315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.088371 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.088463 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:39.088732 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.089322 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.090297 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.090533 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.090650 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.090972 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.091053 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.091136 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.091225 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.091281 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:14:39.099571 19767 raft_consensus.cc:1261] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Refusing update from remote peer 5e347aecfaf84a6a8d11e672ee94572a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:39.099548 19928 raft_consensus.cc:1261] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Refusing update from remote peer 5e347aecfaf84a6a8d11e672ee94572a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:39.100001 20020 consensus_queue.cc:1035] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfa7d9f28734dd5bae2abf7d329b3ea" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:39.100165 20017 consensus_queue.cc:1035] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dcdee7ece734ab285a66baa05bf6591" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:39.134111 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.125222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43589 (local address 127.17.198.2:49437)
0811 22:14:39.129203 (+ 3981us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:39.129247 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:39.129414 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:39.129782 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:39.129784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:39.129805 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:39.129996 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.130005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.131259 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.131265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.132403 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:39.132415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.133118 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.133125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:39.133264 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.133353 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.133737 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.133748 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:14:39.133823 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:14:39.133922 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3794,"thread_start_us":133,"threads_started":1}
I20260811 22:14:39.134090 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.125334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:49437 (local address 127.17.198.1:43589)
0811 22:14:39.125513 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.125519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.129344 (+ 3825us) server_negotiation.cc:407] Connection header received
0811 22:14:39.129595 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.129599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.129668 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.129759 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:39.130122 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.131111 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.132557 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.132993 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.133109 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.133533 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:39.133631 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:39.133731 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.133838 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.133910 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:14:39.163271 19425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:39.163457 19425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:39.166143 19425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 155b23b138374cc1973501d1d28796c3: Sending DeleteTablet for 3 replicas of tablet a85c91f728b74208bfbe043ddac49ac7
I20260811 22:14:39.167102 19585 tablet_service.cc:1514] Processing DeleteTablet for tablet a85c91f728b74208bfbe043ddac49ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:39 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50646
I20260811 22:14:39.167264 19904 tablet_service.cc:1514] Processing DeleteTablet for tablet a85c91f728b74208bfbe043ddac49ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:39 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50428
I20260811 22:14:39.167649 20083 tablet_replica.cc:330] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: stopping tablet replica
I20260811 22:14:39.167871 20083 raft_consensus.cc:2229] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:39.168022 20085 tablet_replica.cc:330] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: stopping tablet replica
I20260811 22:14:39.168181 20083 raft_consensus.cc:2258] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:39.168190 19742 tablet_service.cc:1514] Processing DeleteTablet for tablet a85c91f728b74208bfbe043ddac49ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:39 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:41346
I20260811 22:14:39.168246 20085 raft_consensus.cc:2229] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:39.168620 20088 tablet_replica.cc:330] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: stopping tablet replica
I20260811 22:14:39.168428 20085 raft_consensus.cc:2258] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:39.168780 20088 raft_consensus.cc:2229] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:39.169019 20088 raft_consensus.cc:2258] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:39.169147 20083 ts_tablet_manager.cc:1905] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:39.169912 20085 ts_tablet_manager.cc:1905] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:39.170054 20088 ts_tablet_manager.cc:1905] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:39.173393 20083 ts_tablet_manager.cc:1918] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:39.173393 20088 ts_tablet_manager.cc:1918] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:39.173506 20088 log.cc:1196] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/a85c91f728b74208bfbe043ddac49ac7
I20260811 22:14:39.173487 20085 ts_tablet_manager.cc:1918] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:39.173583 20085 log.cc:1196] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/a85c91f728b74208bfbe043ddac49ac7
I20260811 22:14:39.173727 20083 log.cc:1196] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/a85c91f728b74208bfbe043ddac49ac7
I20260811 22:14:39.174009 20085 ts_tablet_manager.cc:1939] T a85c91f728b74208bfbe043ddac49ac7 P 5cfa7d9f28734dd5bae2abf7d329b3ea: Deleting consensus metadata
I20260811 22:14:39.174209 20083 ts_tablet_manager.cc:1939] T a85c91f728b74208bfbe043ddac49ac7 P 5e347aecfaf84a6a8d11e672ee94572a: Deleting consensus metadata
I20260811 22:14:39.174516 20088 ts_tablet_manager.cc:1939] T a85c91f728b74208bfbe043ddac49ac7 P 2dcdee7ece734ab285a66baa05bf6591: Deleting consensus metadata
I20260811 22:14:39.175076 19413 catalog_manager.cc:4841] TS 5cfa7d9f28734dd5bae2abf7d329b3ea (127.17.198.3:39891): tablet a85c91f728b74208bfbe043ddac49ac7 (table test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]) successfully deleted
I20260811 22:14:39.175320 19413 catalog_manager.cc:4841] TS 5e347aecfaf84a6a8d11e672ee94572a (127.17.198.1:43589): tablet a85c91f728b74208bfbe043ddac49ac7 (table test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]) successfully deleted
I20260811 22:14:39.175472 19413 catalog_manager.cc:4841] TS 2dcdee7ece734ab285a66baa05bf6591 (127.17.198.2:43197): tablet a85c91f728b74208bfbe043ddac49ac7 (table test-table [id=97c14ba50c4a41b6be5cfd12d0db53da]) successfully deleted
Aug 11 22:14:39 dist-test-slave-77v9 krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:39.186853 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.169616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.17.198.1:43589)
0811 22:14:39.169761 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.169766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.175686 (+ 5920us) server_negotiation.cc:407] Connection header received
0811 22:14:39.175935 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.175939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.176002 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.176111 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:39.176950 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.177557 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.178598 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.178856 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.178987 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.182245 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:39.182261 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:39.182285 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:39.182319 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:39.184542 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:39.185241 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:39.185248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:39.185249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:39.185355 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:39.185701 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:39.185710 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:39.185711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:39.186007 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:39.186252 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.186515 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.186667 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:14:39.187455 19637 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60124
I20260811 22:14:39.198750 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:39.189788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.17.198.62:34295)
0811 22:14:39.189943 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:39.189948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:39.189996 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:14:39.190196 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:39.190200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:39.190265 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:39.190369 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:39.191154 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.191983 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.192858 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:39.193252 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:39.193397 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:39.194910 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:39.194915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:39.194918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:39.194953 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:39.196891 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:39.197349 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:39.197353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:39.197355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:39.197477 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:39.197750 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:39.197754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:39.197755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:39.198014 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:39.198162 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:39.198397 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:14:39.198572 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:14:39.199306 19425 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422
I20260811 22:14:39.200739 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19515
I20260811 22:14:39.216956 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19667
I20260811 22:14:39.227478 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19829
I20260811 22:14:39.236646 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 19392
2026-08-11T22:14:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3352 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:39 dist-test-slave-77v9 krb5kdc[20094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:39 dist-test-slave-77v9 krb5kdc[20094](info): set up 2 sockets
Aug 11 22:14:39 dist-test-slave-77v9 krb5kdc[20094](info): commencing operation
krb5kdc: starting...
W20260811 22:14:40.373445 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:40 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:40.521603 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40057
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:40.650453 20107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:40.650801 20107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:40.650851 20107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:40.655252 20107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:40.655342 20107 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:14:40.655360 20107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:40.655378 20107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:40.655396 20107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:40.655412 20107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:40.661242 20107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:40057
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:40.662433 20107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:40.663465 20107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:40.672873 20112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:40.672873 20113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:40.672873 20115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:40.673036 20107 server_base.cc:1034] running on GCE node
I20260811 22:14:40.673854 20107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:40.675122 20107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:40.676322 20107 hybrid_clock.cc:648] HybridClock initialized: now 1786486480676285 us; error 60 us; skew 500 ppm
Aug 11 22:14:40 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486480, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:40.684028 20107 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:40.685286 20107 webserver.cc:460] Webserver started at http://127.17.198.62:35963/ using document root <none> and password file <none>
I20260811 22:14:40.686138 20107 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:40.686272 20107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:40.686573 20107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:40.688982 20107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c98029a237b84a41988f7ff72ed32415"
format_stamp: "Formatted at 2026-08-11 22:14:40 on dist-test-slave-77v9"
server_key: "972d6202e871bb44573a303c64b8d780"
server_key_iv: "3902ad7103bf37e38677a8bd9ea5f1c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:40.689656 20107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c98029a237b84a41988f7ff72ed32415"
format_stamp: "Formatted at 2026-08-11 22:14:40 on dist-test-slave-77v9"
server_key: "972d6202e871bb44573a303c64b8d780"
server_key_iv: "3902ad7103bf37e38677a8bd9ea5f1c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:40.693956 20107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:40.696637 20123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:40.698141 20107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:40.698325 20107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c98029a237b84a41988f7ff72ed32415"
format_stamp: "Formatted at 2026-08-11 22:14:40 on dist-test-slave-77v9"
server_key: "972d6202e871bb44573a303c64b8d780"
server_key_iv: "3902ad7103bf37e38677a8bd9ea5f1c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:40.698472 20107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:40.712062 20107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:40.715692 20107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:40.715948 20107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:40.724758 20107 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:40057
I20260811 22:14:40.724787 20191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:40057 every 8 connection(s)
I20260811 22:14:40.725927 20107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:40.729158 20192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:40.730342 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20107
I20260811 22:14:40.730481 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:40.730792 18200 external_mini_cluster.cc:1428] Setting key bd074828c25b916e7d101a164e92fdaa
I20260811 22:14:40.736011 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Bootstrap starting.
I20260811 22:14:40.738493 20192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:40.739298 20192 log.cc:824] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:14:40 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:40.741850 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: No bootstrap required, opened a new log
I20260811 22:14:40.745144 20192 raft_consensus.cc:348] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } }
I20260811 22:14:40.745452 20192 raft_consensus.cc:374] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:40.745520 20192 raft_consensus.cc:729] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98029a237b84a41988f7ff72ed32415, State: Initialized, Role: FOLLOWER
I20260811 22:14:40.746080 20192 consensus_queue.cc:260] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } }
I20260811 22:14:40.746256 20192 raft_consensus.cc:388] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:40.746351 20192 raft_consensus.cc:482] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:40.746495 20192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:40.747769 20192 raft_consensus.cc:504] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } }
I20260811 22:14:40.748158 20192 leader_election.cc:302] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c98029a237b84a41988f7ff72ed32415; no voters:
I20260811 22:14:40.748543 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:40.732545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.17.198.62:40057)
0811 22:14:40.733092 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:40.733109 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:40.733151 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:14:40.733911 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:40.733931 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:40.734304 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:40.734709 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:40.735693 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:40.736949 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:40.737931 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:40.738469 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:40.738665 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:40.741779 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:40.741787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:40.741806 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:40.741844 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:40.744574 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:40.745236 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:40.745242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:40.745247 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:40.745411 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:40.745983 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:40.745986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:40.745988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:40.746633 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:40.746842 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:40.747145 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:14:40.747525 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":192,"threads_started":1}
I20260811 22:14:40.748579 20192 leader_election.cc:288] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:40.748687 20197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:40.749118 20197 raft_consensus.cc:686] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [term 1 LEADER]: Becoming Leader. State: Replica: c98029a237b84a41988f7ff72ed32415, State: Running, Role: LEADER
I20260811 22:14:40.749459 20197 consensus_queue.cc:237] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } }
I20260811 22:14:40.749924 20192 sys_catalog.cc:564] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:40.750682 20197 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c98029a237b84a41988f7ff72ed32415. Latest consensus state: current_term: 1 leader_uuid: "c98029a237b84a41988f7ff72ed32415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } } }
I20260811 22:14:40.750900 20197 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:40.751374 20206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:40.751572 20199 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c98029a237b84a41988f7ff72ed32415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98029a237b84a41988f7ff72ed32415" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40057 } } }
I20260811 22:14:40.751726 20199 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:40.754719 20206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:40.761013 20206 catalog_manager.cc:1347] Generated new cluster ID: 92281ee9bc9046068e2d043a05d94168
I20260811 22:14:40.761111 20206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:40.794821 20206 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:40.795990 20206 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:40.809010 20206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Generated new TSK 0
I20260811 22:14:40.809921 20206 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:40.849578 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:40057
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:40.979146 20219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:40.979568 20219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:40.979667 20219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:40.984498 20219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:40.984638 20219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:40.984721 20219 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:14:40.984782 20219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:40.984879 20219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:40.990806 20219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:40.992093 20219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:40.993317 20219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:41.004762 20224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.004762 20228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.004794 20225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:41.005071 20219 server_base.cc:1034] running on GCE node
I20260811 22:14:41.005661 20219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:41.006503 20219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:41.007740 20219 hybrid_clock.cc:648] HybridClock initialized: now 1786486481007662 us; error 91 us; skew 500 ppm
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:41.011057 20219 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:41.012070 20219 webserver.cc:460] Webserver started at http://127.17.198.1:37747/ using document root <none> and password file <none>
I20260811 22:14:41.012641 20219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:41.012706 20219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:41.012984 20219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:41.015300 20219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "e1597bff28584e769fa081ca601c5b72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "2cfc906640ec7ea44d48e5b4aefba7d5"
server_key_iv: "70b8e36806cfea6a581a21b938519a45"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.015916 20219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "e1597bff28584e769fa081ca601c5b72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "2cfc906640ec7ea44d48e5b4aefba7d5"
server_key_iv: "70b8e36806cfea6a581a21b938519a45"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.020588 20219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:14:41.023921 20235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.025727 20219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:41.025936 20219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "e1597bff28584e769fa081ca601c5b72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "2cfc906640ec7ea44d48e5b4aefba7d5"
server_key_iv: "70b8e36806cfea6a581a21b938519a45"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.026108 20219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:41.039479 20219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:41.043682 20219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:41.044013 20219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:41.046002 20219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:41.046104 20219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.046226 20219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:41.046288 20219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:41.060753 20219 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:46367
I20260811 22:14:41.060832 20363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:46367 every 8 connection(s)
I20260811 22:14:41.062379 20219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:41.063862 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.046883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:43411 (local address 127.17.198.62:40057)
0811 22:14:41.047101 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.047109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.048721 (+ 1612us) server_negotiation.cc:407] Connection header received
0811 22:14:41.050283 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.050288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.050404 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.050546 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.052182 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.052936 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.054451 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.054734 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.054888 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.058586 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.058591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.058594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.058633 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.060614 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.061572 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.061576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.061577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.061687 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.062302 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.062305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.062306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.062640 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.062736 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.063570 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.063732 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:14:41.064987 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.047834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.1:43411)
0811 22:14:41.048368 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.048560 (+ 192us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.049958 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.050759 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.050766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.051298 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.051970 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.051987 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.053115 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.053118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.054271 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.054283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.054949 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.054953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.055191 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.056054 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.056090 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.058348 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.060905 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.060916 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.060937 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.061423 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.061817 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.061824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.061829 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.062131 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.062777 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.062788 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.063194 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.063470 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.063892 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":150,"threads_started":1}
I20260811 22:14:41.066895 20364 heartbeater.cc:347] Connected to a master server at 127.17.198.62:40057
I20260811 22:14:41.067095 20364 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:41.067749 20364 heartbeater.cc:511] Master 127.17.198.62:40057 requested a full tablet report, sending...
I20260811 22:14:41.069216 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20219
I20260811 22:14:41.069344 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:41.069710 18200 external_mini_cluster.cc:1428] Setting key 06d6ba4c6ac6548e6762cf9e84d18dff
I20260811 22:14:41.069882 20139 ts_manager.cc:194] Registered new tserver with Master: e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367)
I20260811 22:14:41.071300 20139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:43411
I20260811 22:14:41.079751 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.069445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.1:56581)
0811 22:14:41.069731 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.069744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.069828 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.070608 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.070611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.070857 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.070996 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.071005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.071977 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.071984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.073153 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.073169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.073997 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.074000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.074075 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.074687 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.074699 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.075277 (+ 578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.077878 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.077881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.077883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.078149 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.078550 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.078552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.078554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.078645 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.079218 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.079220 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.079316 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.079408 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.079559 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
I20260811 22:14:41.079895 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.070229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:56581 (local address 127.17.198.62:40057)
0811 22:14:41.070358 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.070362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.070378 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:14:41.070417 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.070420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.070468 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.070560 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.071110 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.071788 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.073323 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.073727 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.073836 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.075513 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.075521 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.075528 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.075568 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.077712 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.078281 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.078287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.078292 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.078408 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.078758 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.078764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.078768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.079099 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.079234 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.079516 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.079671 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:41.105605 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:40057
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:41.231144 20373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:41.231534 20373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:41.231590 20373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:41.236042 20373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:41.236143 20373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:41.236188 20373 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:14:41.236203 20373 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:41.236238 20373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:41.242120 20373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:41.243417 20373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:41.244508 20373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:41.256038 20380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.256011 20379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.256081 20382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:41.256106 20373 server_base.cc:1034] running on GCE node
I20260811 22:14:41.256937 20373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:41.257900 20373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:41.259130 20373 hybrid_clock.cc:648] HybridClock initialized: now 1786486481259106 us; error 45 us; skew 500 ppm
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:41.262583 20373 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:41.263829 20373 webserver.cc:460] Webserver started at http://127.17.198.2:33371/ using document root <none> and password file <none>
I20260811 22:14:41.264462 20373 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:41.264569 20373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:41.264904 20373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:41.267256 20373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "1177ff94db254997b2a6915a54686a72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "e8a1a6eda62aa7710317b90d96fb9af8"
server_key_iv: "f8685d04db0a0bb237db47c1fa292927"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.268013 20373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "1177ff94db254997b2a6915a54686a72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "e8a1a6eda62aa7710317b90d96fb9af8"
server_key_iv: "f8685d04db0a0bb237db47c1fa292927"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.272221 20373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 22:14:41.275249 20389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.276787 20373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:14:41.277029 20373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "1177ff94db254997b2a6915a54686a72"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "e8a1a6eda62aa7710317b90d96fb9af8"
server_key_iv: "f8685d04db0a0bb237db47c1fa292927"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.277189 20373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:41.295647 20373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:41.299602 20373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:41.299911 20373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:41.301877 20373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:41.301990 20373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.302078 20373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:41.302129 20373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:41.315085 20373 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:46347
I20260811 22:14:41.315083 20517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:46347 every 8 connection(s)
I20260811 22:14:41.316349 20373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:41.318035 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.302914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:53039 (local address 127.17.198.62:40057)
0811 22:14:41.303132 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.303136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.304466 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 22:14:41.305696 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.305700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.305767 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.305886 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.307185 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.307761 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.309349 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.309827 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.309996 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.313385 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.313389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.313393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.313462 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.315467 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.316132 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.316135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.316136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.316239 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.316648 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.316651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.316653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.316979 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.317091 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.317768 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.317892 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 22:14:41.318944 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.303573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.2:53039)
0811 22:14:41.304225 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.304327 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.305422 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.306075 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.306082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.306470 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.307019 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.307032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.307911 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.307917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.309182 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.309197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.310052 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.310059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.310239 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.311311 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.311341 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.313168 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.315600 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.315606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.315621 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.315989 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.316355 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.316362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.316367 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.316548 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.317153 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.317159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.317501 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.317675 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.317967 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":144,"threads_started":1}
I20260811 22:14:41.320313 20518 heartbeater.cc:347] Connected to a master server at 127.17.198.62:40057
I20260811 22:14:41.320514 20518 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:41.321146 20518 heartbeater.cc:511] Master 127.17.198.62:40057 requested a full tablet report, sending...
I20260811 22:14:41.322471 20141 ts_manager.cc:194] Registered new tserver with Master: 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347)
I20260811 22:14:41.323264 20141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:53039
I20260811 22:14:41.324891 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20373
I20260811 22:14:41.325026 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:41.325317 18200 external_mini_cluster.cc:1428] Setting key c28b8cc78c008d5b293d9327bcd1b0d2
I20260811 22:14:41.332691 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.322815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:52483 (local address 127.17.198.62:40057)
0811 22:14:41.322951 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.322956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.323835 (+ 879us) server_negotiation.cc:407] Connection header received
0811 22:14:41.323995 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.323998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.324048 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.324134 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.324720 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.325519 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.326599 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.326958 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.327090 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.328718 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.328722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.328724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.328747 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.330665 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.331207 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.331211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.331213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.331310 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.331666 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.331672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.331674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.331942 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.332070 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.332371 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.332511 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:14:41.332706 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.322739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.2:52483)
0811 22:14:41.323735 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.323752 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.323882 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.324200 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.324203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.324416 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.324571 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.324581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.325670 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.325674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.326442 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.326453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.327124 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.327130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.327229 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.328026 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.328056 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.328564 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.330808 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.330812 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.330814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.331088 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.331417 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.331421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.331423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.331526 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.332057 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.332061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.332183 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.332307 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.332503 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:41.361335 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:40057
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:41.482863 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:41.483260 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:41.483321 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:41.487692 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:41.487776 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:41.487825 20527 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:14:41.487840 20527 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:14:41.487874 20527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:41.493616 20527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:41.494794 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:41.495827 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:14:41.506759 20527 server_base.cc:1034] running on GCE node
W20260811 22:14:41.506673 20536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.506654 20533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:41.506690 20532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:41.507436 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:41.508198 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:41.509414 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786486481509371 us; error 55 us; skew 500 ppm
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:41.512933 20527 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:41.514139 20527 webserver.cc:460] Webserver started at http://127.17.198.3:45317/ using document root <none> and password file <none>
I20260811 22:14:41.514813 20527 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:41.514917 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:41.515182 20527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:41.517582 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "381c4b8188a1477f9dc5c73c6a537126"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "1eef60549d9f2a817670cb2f41e40405"
server_key_iv: "cf9e9ba594c4db5f6ad52dd0b8fc09b6"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.518247 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "381c4b8188a1477f9dc5c73c6a537126"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "1eef60549d9f2a817670cb2f41e40405"
server_key_iv: "cf9e9ba594c4db5f6ad52dd0b8fc09b6"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.522909 20527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:14:41.525986 20544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.527536 20527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:41.527737 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "381c4b8188a1477f9dc5c73c6a537126"
format_stamp: "Formatted at 2026-08-11 22:14:41 on dist-test-slave-77v9"
server_key: "1eef60549d9f2a817670cb2f41e40405"
server_key_iv: "cf9e9ba594c4db5f6ad52dd0b8fc09b6"
server_key_version: "encryptionkey@0"
I20260811 22:14:41.527891 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:41.562383 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:41.566080 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:41.566330 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:41.568136 20527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:41.568223 20527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:41.568293 20527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:41.568338 20527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:41.580410 20527 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:38005
I20260811 22:14:41.580507 20674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:38005 every 8 connection(s)
I20260811 22:14:41.581880 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:41.584949 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.569337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:38229 (local address 127.17.198.62:40057)
0811 22:14:41.569508 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.569512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.570687 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 22:14:41.572179 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.572183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.572272 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.572390 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.574048 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.574819 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.576535 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.576953 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.577099 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.580669 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.580674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.580678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.580712 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.582633 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.583313 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.583316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.583317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.583409 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.583766 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.583768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.583769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.583987 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.584067 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.584624 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.584740 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:14:41.585870 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.569925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.3:38229)
0811 22:14:41.570491 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.570542 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.571890 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.572629 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.572639 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.573158 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.573844 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.573861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.574969 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.574975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.576370 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.576384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.577156 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.577161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.577387 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.578202 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.578230 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.580437 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.582814 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.582821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.582836 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.583201 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.583532 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.583534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.583536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.583679 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.584110 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.584118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.584391 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.584559 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.584905 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":149,"threads_started":1}
I20260811 22:14:41.588644 20675 heartbeater.cc:347] Connected to a master server at 127.17.198.62:40057
I20260811 22:14:41.588907 20675 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:41.589558 20675 heartbeater.cc:511] Master 127.17.198.62:40057 requested a full tablet report, sending...
I20260811 22:14:41.590413 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20527
I20260811 22:14:41.590551 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:41.590734 20139 ts_manager.cc:194] Registered new tserver with Master: 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:41.590904 18200 external_mini_cluster.cc:1428] Setting key 34c54a7eb7b500ab5c5ae1056bce2e2f
I20260811 22:14:41.591508 20139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:38229
I20260811 22:14:41.593545 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:41.599120 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.590301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:50241 (local address 127.17.198.62:40057)
0811 22:14:41.590445 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.590449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.590743 (+ 294us) server_negotiation.cc:407] Connection header received
0811 22:14:41.590861 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.590864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.590909 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.590993 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.591604 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.592245 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.593220 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.593479 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.593590 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.595053 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.595060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.595066 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.595103 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.597043 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.597722 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.597725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.597726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.597814 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.598154 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.598156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.598157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.598382 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.598464 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.598868 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.598991 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:14:41.599326 20681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.590205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.17.198.3:50241)
0811 22:14:41.590630 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.590646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.590767 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.591045 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.591048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.591300 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:41.591463 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.591472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.592372 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.592375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.593092 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.593102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.593611 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.593615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.593723 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.594414 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:41.594431 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:41.594884 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:41.597180 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.597186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.597188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.597566 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.597926 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:41.597932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:41.597937 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:41.598052 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:41.598505 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:41.598510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:41.598647 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.598772 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.598988 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:14:41 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:41.625353 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.615782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.17.198.62:40057)
0811 22:14:41.615970 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.615973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.615987 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:41.616032 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.616035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.616082 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.616192 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:41.617076 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.617619 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.618515 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.618748 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.618872 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.621827 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:41.621833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:41.621836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:41.621876 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:41.623350 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.623936 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.623940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.623942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.624040 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:41.624335 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:41.624338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:41.624339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:41.624573 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:41.624689 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.625076 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.625213 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:14:41.629011 20139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:41.632431 20139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:41.649412 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.642544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38005 (local address 127.0.0.1:59572)
0811 22:14:41.643221 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.643238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.643412 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.644502 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.644506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.644525 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:41.644640 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.644648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.646057 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.646062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.647569 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.647578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.648710 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.648714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.648882 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.648936 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.649060 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.649203 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":156,"spinlock_wait_cycles":768,"thread_start_us":122,"threads_started":1}
I20260811 22:14:41.649585 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.642548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53648 (local address 127.17.198.1:46367)
0811 22:14:41.642896 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.642903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.643564 (+ 661us) server_negotiation.cc:407] Connection header received
0811 22:14:41.643649 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.643654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.643874 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.644039 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:41.644542 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.645828 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.648149 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.648908 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.649048 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.649193 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.649289 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.649392 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":137,"threads_started":1}
I20260811 22:14:41.649818 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.643724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.17.198.2:46347)
0811 22:14:41.644293 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.644308 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.644333 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:14:41.644419 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.644423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.644627 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.644785 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:41.645141 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.645995 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.647928 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.648742 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.648887 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.649057 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.649395 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.649499 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":206,"threads_started":1}
I20260811 22:14:41.650043 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.643155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.17.198.3:38005)
0811 22:14:41.643892 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:41.643903 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:41.643928 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:14:41.644002 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:41.644007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:41.644347 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:41.644514 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:41.644776 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.645893 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.647697 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.648555 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.648675 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.648893 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:41.649713 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 22:14:41.649828 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":133,"threads_started":1}
I20260811 22:14:41.650108 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.643247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.0.0.1:49416)
0811 22:14:41.643662 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.643675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.643762 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.644899 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.644902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.644919 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:41.645016 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.645020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.646138 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.646142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.647778 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.647789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.649110 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.649113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.649185 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.649213 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.649276 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.649985 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":791,"thread_start_us":122,"threads_started":1}
I20260811 22:14:41.651139 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:41.642309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46367 (local address 127.0.0.1:53648)
0811 22:14:41.643172 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:41.643234 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:41.643412 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:41.644209 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:41.644215 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:41.644237 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:41.644405 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.644415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.645967 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.645971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.648017 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:41.648025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:41.649046 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:41.649050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:41.649150 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:41.649173 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:14:41.649257 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:14:41.649361 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":159,"spinlock_wait_cycles":13312,"thread_start_us":88,"threads_started":1}
I20260811 22:14:41.653215 20603 tablet_service.cc:1467] Processing CreateTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b (DEFAULT_TABLE table=test-table [id=a2fadb02a27f4effbfee1c7dd703161b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:41.653566 20290 tablet_service.cc:1467] Processing CreateTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b (DEFAULT_TABLE table=test-table [id=a2fadb02a27f4effbfee1c7dd703161b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:41.654465 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e720db2ba604b99a0f8f960a116cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:41.654610 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e720db2ba604b99a0f8f960a116cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:41.659032 20448 tablet_service.cc:1467] Processing CreateTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b (DEFAULT_TABLE table=test-table [id=a2fadb02a27f4effbfee1c7dd703161b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:41.660364 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e720db2ba604b99a0f8f960a116cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:41.661243 20702 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Bootstrap starting.
I20260811 22:14:41.663605 20702 tablet_bootstrap.cc:654] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:41.664444 20702 log.cc:824] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:41.667199 20702 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: No bootstrap required, opened a new log
I20260811 22:14:41.667439 20702 ts_tablet_manager.cc:1397] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 22:14:41.667670 20704 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Bootstrap starting.
I20260811 22:14:41.670606 20704 tablet_bootstrap.cc:654] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:41.671598 20704 log.cc:824] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:41.671635 20707 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Bootstrap starting.
I20260811 22:14:41.672019 20702 raft_consensus.cc:348] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.672392 20702 raft_consensus.cc:374] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:41.672487 20702 raft_consensus.cc:729] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Initialized, Role: FOLLOWER
I20260811 22:14:41.673028 20702 consensus_queue.cc:260] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.674214 20707 tablet_bootstrap.cc:654] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:41.674330 20364 heartbeater.cc:503] Master 127.17.198.62:40057 was elected leader, sending a full tablet report...
I20260811 22:14:41.674511 20702 ts_tablet_manager.cc:1428] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:14:41.674651 20704 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: No bootstrap required, opened a new log
I20260811 22:14:41.674868 20704 ts_tablet_manager.cc:1397] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 22:14:41.675230 20707 log.cc:824] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:41.677855 20707 tablet_bootstrap.cc:492] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: No bootstrap required, opened a new log
I20260811 22:14:41.678128 20707 ts_tablet_manager.cc:1397] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:14:41.678645 20704 raft_consensus.cc:348] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.678931 20704 raft_consensus.cc:374] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:41.679021 20704 raft_consensus.cc:729] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Initialized, Role: FOLLOWER
I20260811 22:14:41.679512 20704 consensus_queue.cc:260] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.680397 20675 heartbeater.cc:503] Master 127.17.198.62:40057 was elected leader, sending a full tablet report...
I20260811 22:14:41.680634 20704 ts_tablet_manager.cc:1428] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 22:14:41.682310 20707 raft_consensus.cc:348] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.682646 20707 raft_consensus.cc:374] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:41.682708 20707 raft_consensus.cc:729] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Initialized, Role: FOLLOWER
I20260811 22:14:41.683162 20707 consensus_queue.cc:260] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:41.683923 20518 heartbeater.cc:503] Master 127.17.198.62:40057 was elected leader, sending a full tablet report...
I20260811 22:14:41.684338 20707 ts_tablet_manager.cc:1428] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 22:14:41.814255 20365 tablet.cc:2369] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:41.817693 20519 tablet.cc:2369] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:41.833209 20676 tablet.cc:2369] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:42.015686 20711 raft_consensus.cc:482] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:42.015960 20711 raft_consensus.cc:504] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:42.017236 20711 leader_election.cc:288] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347)
I20260811 22:14:42.021624 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.017800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:42879 (local address 127.17.198.2:46347)
0811 22:14:42.017923 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.017927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.017942 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:14:42.018019 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.018021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.018060 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.018164 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.018446 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.018955 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.020239 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.021010 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.021152 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.021208 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.021412 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.021485 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:14:42.021624 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.017543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.17.198.3:42879)
0811 22:14:42.017689 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.017707 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.017899 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.018177 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.018180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.018198 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.018322 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.018328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.019071 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.019077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.020060 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.020075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.021153 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.021156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.021262 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.021287 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.021400 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.021482 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":71}
I20260811 22:14:42.022377 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e720db2ba604b99a0f8f960a116cd7b" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72" is_pre_election: true
I20260811 22:14:42.022754 20470 raft_consensus.cc:2454] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 0.
I20260811 22:14:42.022918 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.017556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46367 (local address 127.17.198.3:34009)
0811 22:14:42.018025 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.018043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.018185 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.018486 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.018489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.018507 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.019056 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.019065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.019951 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.019955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.021433 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.021444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.022443 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.022448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.022564 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.022586 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.022670 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.022750 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":47,"thread_start_us":114,"threads_started":1}
I20260811 22:14:42.022938 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.017654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:34009 (local address 127.17.198.1:46367)
0811 22:14:42.017787 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.017792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.018146 (+ 354us) server_negotiation.cc:407] Connection header received
0811 22:14:42.018310 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.018314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.018373 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.018472 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.019201 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.019783 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.021573 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.022279 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.022443 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.022498 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.022710 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.022808 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:14:42.023353 20545 leader_election.cc:302] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, 381c4b8188a1477f9dc5c73c6a537126; no voters:
I20260811 22:14:42.023615 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e720db2ba604b99a0f8f960a116cd7b" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72" is_pre_election: true
I20260811 22:14:42.023643 20711 raft_consensus.cc:2781] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:42.023751 20711 raft_consensus.cc:482] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:42.023819 20711 raft_consensus.cc:3037] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.023965 20314 raft_consensus.cc:2454] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 0.
I20260811 22:14:42.025101 20711 raft_consensus.cc:504] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:42.025550 20711 leader_election.cc:288] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 election: Requested vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347)
I20260811 22:14:42.025985 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e720db2ba604b99a0f8f960a116cd7b" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72"
I20260811 22:14:42.026139 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e720db2ba604b99a0f8f960a116cd7b" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72"
I20260811 22:14:42.026228 20470 raft_consensus.cc:3037] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.026346 20314 raft_consensus.cc:3037] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.027565 20314 raft_consensus.cc:2454] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 1.
I20260811 22:14:42.027578 20470 raft_consensus.cc:2454] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 1.
I20260811 22:14:42.027956 20547 leader_election.cc:302] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 381c4b8188a1477f9dc5c73c6a537126, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:42.028178 20711 raft_consensus.cc:2781] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:42.028383 20711 raft_consensus.cc:686] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 1 LEADER]: Becoming Leader. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Running, Role: LEADER
I20260811 22:14:42.028771 20711 consensus_queue.cc:237] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } }
I20260811 22:14:42.032557 20139 catalog_manager.cc:5495] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 reported cstate change: term changed from 0 to 1, leader changed from <none> to 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "381c4b8188a1477f9dc5c73c6a537126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:14:42 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486480, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:42.133549 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.123232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.17.198.62:40057)
0811 22:14:42.123414 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.123419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.123593 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:14:42.123774 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.123777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.123828 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.123927 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:42.124709 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.125408 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.126302 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.126559 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.126683 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.129645 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:42.129650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:42.129652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:42.129682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:42.131466 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:42.132049 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:42.132054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:42.132057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:42.132143 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:42.132444 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:42.132449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:42.132452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:42.132705 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:42.132845 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.133191 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.133404 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":59}
I20260811 22:14:42.133549 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.123128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40057 (local address 127.0.0.1:44994)
0811 22:14:42.123478 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.123500 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.123655 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.123948 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.123951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.124455 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:42.124564 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.124572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.125529 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.125536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.126168 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.126178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.126693 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.126697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.126784 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.127520 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:42.127535 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:42.129475 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:42.131579 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:42.131595 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:42.131603 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:42.131942 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:42.132250 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:42.132253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:42.132255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:42.132351 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:42.132847 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:42.132856 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:42.132997 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.133126 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.133326 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
I20260811 22:14:42.138423 20139 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:44994:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:14:42.139287 20139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:42.144910 20448 tablet_service.cc:1467] Processing CreateTablet for tablet 509bb320222b42798ced769b39c0707f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d3f27dadd96424ebf89f337ca8bbe49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:42.144712 20290 tablet_service.cc:1467] Processing CreateTablet for tablet 509bb320222b42798ced769b39c0707f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d3f27dadd96424ebf89f337ca8bbe49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:42.145040 20603 tablet_service.cc:1467] Processing CreateTablet for tablet 509bb320222b42798ced769b39c0707f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d3f27dadd96424ebf89f337ca8bbe49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:14:42.145646 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509bb320222b42798ced769b39c0707f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.145645 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509bb320222b42798ced769b39c0707f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.145814 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509bb320222b42798ced769b39c0707f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.148782 20702 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: Bootstrap starting.
I20260811 22:14:42.148854 20707 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: Bootstrap starting.
I20260811 22:14:42.149690 20702 tablet_bootstrap.cc:654] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.149776 20707 tablet_bootstrap.cc:654] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.150527 20704 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: Bootstrap starting.
I20260811 22:14:42.151424 20702 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: No bootstrap required, opened a new log
I20260811 22:14:42.151531 20702 ts_tablet_manager.cc:1397] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:14:42.151690 20704 tablet_bootstrap.cc:654] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.152053 20702 raft_consensus.cc:348] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.152202 20702 raft_consensus.cc:374] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.152249 20702 raft_consensus.cc:729] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.152398 20702 consensus_queue.cc:260] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.152701 20709 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.152884 20709 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.153219 20704 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: No bootstrap required, opened a new log
I20260811 22:14:42.153340 20704 ts_tablet_manager.cc:1397] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:42.153520 20702 ts_tablet_manager.cc:1428] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 22:14:42.153854 20704 raft_consensus.cc:348] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.153990 20704 raft_consensus.cc:374] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.154027 20704 raft_consensus.cc:729] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.154152 20704 consensus_queue.cc:260] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.154479 20704 ts_tablet_manager.cc:1428] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:14:42.154762 20711 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.154906 20711 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.156002 20707 tablet_bootstrap.cc:492] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: No bootstrap required, opened a new log
I20260811 22:14:42.156118 20707 ts_tablet_manager.cc:1397] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:14:42.156639 20707 raft_consensus.cc:348] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.156771 20707 raft_consensus.cc:374] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.156889 20707 raft_consensus.cc:729] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.157022 20707 consensus_queue.cc:260] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.157325 20707 ts_tablet_manager.cc:1428] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:14:42.157660 20712 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.157835 20712 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.213302 20712 raft_consensus.cc:482] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:42.213518 20712 raft_consensus.cc:504] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.214864 20712 leader_election.cc:288] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:42.220036 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.215257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46367 (local address 127.17.198.2:40605)
0811 22:14:42.215688 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.215706 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.215829 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.216203 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.216206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.216224 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.216411 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.216419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.217575 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.217578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.218662 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.218673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.219553 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.219556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.219637 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.219657 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.219779 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.219858 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260811 22:14:42.220081 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.215665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:40605 (local address 127.17.198.1:46367)
0811 22:14:42.215810 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.215873 (+ 63us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.215902 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:14:42.215992 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.215996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.216051 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.216160 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.216568 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.217152 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.218806 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.219400 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.219534 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.219589 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.219812 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.219904 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:14:42.220521 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.215242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38005 (local address 127.17.198.2:55891)
0811 22:14:42.215584 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.215602 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.215748 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.216417 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.216420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.216438 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.217140 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.217150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.218194 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.218198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.219094 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.219102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.220043 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.220046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.220189 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.220216 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.220279 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.220360 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
I20260811 22:14:42.220667 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509bb320222b42798ced769b39c0707f" candidate_uuid: "1177ff94db254997b2a6915a54686a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72" is_pre_election: true
I20260811 22:14:42.220865 20314 raft_consensus.cc:2454] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1177ff94db254997b2a6915a54686a72 in term 0.
I20260811 22:14:42.221046 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.215766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:55891 (local address 127.17.198.3:38005)
0811 22:14:42.216093 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.216111 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.216127 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:14:42.216183 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.216186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.216262 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.216356 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.217437 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.218059 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.219224 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.219888 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.220613 (+ 725us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.220682 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.220727 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.220883 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":110,"threads_started":1}
I20260811 22:14:42.221287 20392 leader_election.cc:302] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:42.221706 20712 raft_consensus.cc:2781] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:42.221798 20712 raft_consensus.cc:482] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:42.221706 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509bb320222b42798ced769b39c0707f" candidate_uuid: "1177ff94db254997b2a6915a54686a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126" is_pre_election: true
I20260811 22:14:42.221872 20712 raft_consensus.cc:3037] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.222008 20612 raft_consensus.cc:2454] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1177ff94db254997b2a6915a54686a72 in term 0.
I20260811 22:14:42.223168 20712 raft_consensus.cc:504] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.223528 20712 leader_election.cc:288] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [CANDIDATE]: Term 1 election: Requested vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:42.223999 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509bb320222b42798ced769b39c0707f" candidate_uuid: "1177ff94db254997b2a6915a54686a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72"
I20260811 22:14:42.224133 20314 raft_consensus.cc:3037] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.225487 20314 raft_consensus.cc:2454] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1177ff94db254997b2a6915a54686a72 in term 1.
I20260811 22:14:42.225883 20392 leader_election.cc:302] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:42.225863 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509bb320222b42798ced769b39c0707f" candidate_uuid: "1177ff94db254997b2a6915a54686a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126"
I20260811 22:14:42.226009 20612 raft_consensus.cc:3037] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.226133 20712 raft_consensus.cc:2781] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:42.226359 20712 raft_consensus.cc:686] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [term 1 LEADER]: Becoming Leader. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Running, Role: LEADER
I20260811 22:14:42.226794 20712 consensus_queue.cc:237] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.227417 20612 raft_consensus.cc:2454] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1177ff94db254997b2a6915a54686a72 in term 1.
I20260811 22:14:42.229576 20729 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: TxnStatusTablet state changed. Reason: New leader 1177ff94db254997b2a6915a54686a72. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.229719 20729 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:42.230123 20731 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:14:42.230400 20731 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:14:42.231014 20141 catalog_manager.cc:5495] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1177ff94db254997b2a6915a54686a72 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260811 22:14:42.257840 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.253034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.0.0.1:49424)
0811 22:14:42.253274 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.253293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.253455 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.253909 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.253912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.253919 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:42.254023 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.254028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.255101 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.255104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.255792 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.255801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.256419 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.256422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.256494 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.256550 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.257500 (+ 950us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.257520 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.257591 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.257684 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:14:42.257843 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.253314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49424 (local address 127.17.198.2:46347)
0811 22:14:42.253443 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.253448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.253466 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:14:42.253638 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.253641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.253712 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.253823 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.254142 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.254943 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.255932 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.256281 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.256398 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.256716 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.257357 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.257478 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.257639 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.257704 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:14:42.264145 20612 raft_consensus.cc:1261] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Refusing update from remote peer 1177ff94db254997b2a6915a54686a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.264146 20314 raft_consensus.cc:1261] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Refusing update from remote peer 1177ff94db254997b2a6915a54686a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.264883 20729 consensus_queue.cc:1035] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.265082 20712 consensus_queue.cc:1035] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.268613 20711 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: TxnStatusTablet state changed. Reason: New leader 1177ff94db254997b2a6915a54686a72. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.268769 20711 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.269598 20709 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: TxnStatusTablet state changed. Reason: New leader 1177ff94db254997b2a6915a54686a72. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.269762 20709 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.273059 20712 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.273193 20712 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:42.273339 20734 mvcc.cc:205] Tried to move back new op lower bound from 7317448631346085888 to 7317448631204577280. Current Snapshot: MvccSnapshot[applied={T|T < 7317448631346085888}]
I20260811 22:14:42.273802 20711 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.273926 20711 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 381c4b8188a1477f9dc5c73c6a537126: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.274181 20729 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.274281 20729 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P 1177ff94db254997b2a6915a54686a72: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:14:42.276492 20709 tablet_replica.cc:439] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1177ff94db254997b2a6915a54686a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } } }
I20260811 22:14:42.276623 20709 tablet_replica.cc:442] T 509bb320222b42798ced769b39c0707f P e1597bff28584e769fa081ca601c5b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:14:42.277231 20735 mvcc.cc:205] Tried to move back new op lower bound from 7317448631346085888 to 7317448631204577280. Current Snapshot: MvccSnapshot[applied={T|T < 7317448631346085888}]
I20260811 22:14:42.277271 20736 mvcc.cc:205] Tried to move back new op lower bound from 7317448631346085888 to 7317448631204577280. Current Snapshot: MvccSnapshot[applied={T|T < 7317448631346085888}]
I20260811 22:14:42.286057 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.281546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.17.198.3:38005)
0811 22:14:42.281676 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.281680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.281849 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:14:42.281983 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.281986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.282046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.282148 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.282413 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.283032 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.284295 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.284542 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.284642 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.285023 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.285525 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.285628 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.285730 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.285788 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 22:14:42.298887 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.294717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:34443 (local address 127.17.198.2:46347)
0811 22:14:42.294882 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.294887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.295180 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:14:42.295379 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.295387 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.295437 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.295532 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.295824 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.296420 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.297597 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.297838 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.297969 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.298355 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.298445 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.298534 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.298669 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.298725 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:14:42.298944 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.294598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.17.198.3:34443)
0811 22:14:42.295081 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.295095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.295246 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.295548 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.295551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.295560 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:42.295706 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.295712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.296550 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.296555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.297454 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.297465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.297990 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.297994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.298096 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.298147 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.298543 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.298553 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.298644 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.298732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":140,"threads_started":1}
I20260811 22:14:42.308161 20470 raft_consensus.cc:1261] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Refusing update from remote peer 381c4b8188a1477f9dc5c73c6a537126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.308547 20314 raft_consensus.cc:1261] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Refusing update from remote peer 381c4b8188a1477f9dc5c73c6a537126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.308919 20714 consensus_queue.cc:1035] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.309156 20711 consensus_queue.cc:1035] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.340927 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.336881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:43519 (local address 127.17.198.3:38005)
0811 22:14:42.337021 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.337026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.337043 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:42.337104 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.337108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.337213 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.337302 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.337740 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.338321 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.339399 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.339741 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.339825 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.340298 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.340412 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.340468 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.340658 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.340725 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:14:42.341391 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.336224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38005 (local address 127.17.198.2:43519)
0811 22:14:42.336613 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.336632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.336737 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.337485 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.337488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.337498 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:42.337623 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.337631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.338476 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.338480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.339192 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.339234 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.339955 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.339959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.340060 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.340125 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.340538 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.340544 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.341110 (+ 566us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.341225 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":153,"threads_started":1}
I20260811 22:14:42.371845 20138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:42.372089 20138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:42.375891 20138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Sending DeleteTablet for 3 replicas of tablet 0e720db2ba604b99a0f8f960a116cd7b
I20260811 22:14:42.377388 20290 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:42 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260811 22:14:42.377523 20448 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:42 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:49416
I20260811 22:14:42.378364 20755 tablet_replica.cc:330] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: stopping tablet replica
I20260811 22:14:42.378443 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e720db2ba604b99a0f8f960a116cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:42 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:59572
I20260811 22:14:42.378810 20755 raft_consensus.cc:2229] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:42.379091 20755 raft_consensus.cc:2258] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:42.380016 20138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:42.380573 20138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:42.381028 20755 ts_tablet_manager.cc:1905] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:42.385234 20755 ts_tablet_manager.cc:1918] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:42.385408 20755 log.cc:1196] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/0e720db2ba604b99a0f8f960a116cd7b
I20260811 22:14:42.385842 20755 ts_tablet_manager.cc:1939] T 0e720db2ba604b99a0f8f960a116cd7b P e1597bff28584e769fa081ca601c5b72: Deleting consensus metadata
I20260811 22:14:42.387593 20125 catalog_manager.cc:4841] TS e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367): tablet 0e720db2ba604b99a0f8f960a116cd7b (table test-table [id=a2fadb02a27f4effbfee1c7dd703161b]) successfully deleted
I20260811 22:14:42.388135 20290 tablet_service.cc:1467] Processing CreateTablet for tablet b83788aee4404493b59cc4b1d8775cb8 (DEFAULT_TABLE table=test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:42.388216 20603 tablet_service.cc:1467] Processing CreateTablet for tablet b83788aee4404493b59cc4b1d8775cb8 (DEFAULT_TABLE table=test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:42.388486 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83788aee4404493b59cc4b1d8775cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.388581 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83788aee4404493b59cc4b1d8775cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.391484 20702 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Bootstrap starting.
I20260811 22:14:42.392832 20702 tablet_bootstrap.cc:654] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.392905 20448 tablet_service.cc:1467] Processing CreateTablet for tablet b83788aee4404493b59cc4b1d8775cb8 (DEFAULT_TABLE table=test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:42.393241 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83788aee4404493b59cc4b1d8775cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:42.393704 20704 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Bootstrap starting.
I20260811 22:14:42.394928 20704 tablet_bootstrap.cc:654] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.397809 20702 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: No bootstrap required, opened a new log
I20260811 22:14:42.397944 20702 ts_tablet_manager.cc:1397] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:14:42.398393 20707 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Bootstrap starting.
I20260811 22:14:42.398535 20702 raft_consensus.cc:348] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.398680 20702 raft_consensus.cc:374] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.398720 20702 raft_consensus.cc:729] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.398896 20702 consensus_queue.cc:260] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.399224 20702 ts_tablet_manager.cc:1428] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:14:42.399605 20707 tablet_bootstrap.cc:654] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:42.402415 20704 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: No bootstrap required, opened a new log
I20260811 22:14:42.402562 20704 ts_tablet_manager.cc:1397] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.002s
I20260811 22:14:42.403178 20704 raft_consensus.cc:348] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.403308 20707 tablet_bootstrap.cc:492] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: No bootstrap required, opened a new log
I20260811 22:14:42.403342 20704 raft_consensus.cc:374] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.403384 20704 raft_consensus.cc:729] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.403404 20707 ts_tablet_manager.cc:1397] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:14:42.403546 20704 consensus_queue.cc:260] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.403891 20704 ts_tablet_manager.cc:1428] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:14:42.403908 20707 raft_consensus.cc:348] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.404038 20707 raft_consensus.cc:374] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:42.404070 20707 raft_consensus.cc:729] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Initialized, Role: FOLLOWER
I20260811 22:14:42.404196 20707 consensus_queue.cc:260] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.404451 20707 ts_tablet_manager.cc:1428] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:42.407285 20757 tablet_replica.cc:330] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: stopping tablet replica
I20260811 22:14:42.407800 20757 raft_consensus.cc:2229] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:42.408334 20757 raft_consensus.cc:2258] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:42.408767 20756 tablet_replica.cc:330] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: stopping tablet replica
I20260811 22:14:42.409168 20756 raft_consensus.cc:2229] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:42.409360 20756 raft_consensus.cc:2258] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:42.410454 20757 ts_tablet_manager.cc:1905] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:42.410630 20756 ts_tablet_manager.cc:1905] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:42.414394 20756 ts_tablet_manager.cc:1918] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:42.414562 20756 log.cc:1196] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/0e720db2ba604b99a0f8f960a116cd7b
I20260811 22:14:42.414460 20757 ts_tablet_manager.cc:1918] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:42.414784 20757 log.cc:1196] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/0e720db2ba604b99a0f8f960a116cd7b
I20260811 22:14:42.414942 20756 ts_tablet_manager.cc:1939] T 0e720db2ba604b99a0f8f960a116cd7b P 1177ff94db254997b2a6915a54686a72: Deleting consensus metadata
I20260811 22:14:42.415308 20757 ts_tablet_manager.cc:1939] T 0e720db2ba604b99a0f8f960a116cd7b P 381c4b8188a1477f9dc5c73c6a537126: Deleting consensus metadata
I20260811 22:14:42.416352 20124 catalog_manager.cc:4841] TS 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347): tablet 0e720db2ba604b99a0f8f960a116cd7b (table test-table [id=a2fadb02a27f4effbfee1c7dd703161b]) successfully deleted
I20260811 22:14:42.416687 20127 catalog_manager.cc:4841] TS 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005): tablet 0e720db2ba604b99a0f8f960a116cd7b (table test-table [id=a2fadb02a27f4effbfee1c7dd703161b]) successfully deleted
I20260811 22:14:42.622318 20709 raft_consensus.cc:482] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:42.622494 20709 raft_consensus.cc:504] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.623768 20709 leader_election.cc:288] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:42.629135 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.624136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:49889 (local address 127.17.198.2:46347)
0811 22:14:42.624360 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.624364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.625039 (+ 675us) server_negotiation.cc:407] Connection header received
0811 22:14:42.625238 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.625241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.625291 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.625391 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.625964 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.626464 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.627840 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.628271 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.628381 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.628429 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.628847 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.628995 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:14:42.629208 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.624241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:49665 (local address 127.17.198.3:38005)
0811 22:14:42.624380 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.624384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.625011 (+ 627us) server_negotiation.cc:407] Connection header received
0811 22:14:42.625202 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.625209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.625275 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.625353 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:42.625777 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.626457 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.627782 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.628544 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.628690 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.628756 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.628974 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.629066 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:14:42.629242 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.624006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.17.198.1:49889)
0811 22:14:42.624789 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.624835 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.625090 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.625435 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.625438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.625482 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.625776 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.625785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.626663 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.626666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.627663 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.627673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.628498 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.628503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.628661 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.628690 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.628780 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.628937 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":51,"spinlock_wait_cycles":4224,"thread_start_us":521,"threads_started":1}
I20260811 22:14:42.629256 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.624117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38005 (local address 127.17.198.1:49665)
0811 22:14:42.624895 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.624908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.625060 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.625398 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.625401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.625417 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:42.625606 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.625613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.626620 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.626625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.627622 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.627633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.628692 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.628695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.628809 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.628857 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.628937 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.629009 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":58,"spinlock_wait_cycles":25728,"thread_start_us":399,"threads_started":1}
I20260811 22:14:42.630072 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83788aee4404493b59cc4b1d8775cb8" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72" is_pre_election: true
I20260811 22:14:42.630353 20470 raft_consensus.cc:2454] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 0.
I20260811 22:14:42.630441 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83788aee4404493b59cc4b1d8775cb8" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126" is_pre_election: true
I20260811 22:14:42.630585 20612 raft_consensus.cc:2454] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 0.
I20260811 22:14:42.630921 20236 leader_election.cc:302] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:42.631279 20709 raft_consensus.cc:2781] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:42.631343 20709 raft_consensus.cc:482] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:42.631372 20709 raft_consensus.cc:3037] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.633308 20709 raft_consensus.cc:504] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.633790 20709 leader_election.cc:288] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 election: Requested vote from peers 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:42.634359 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83788aee4404493b59cc4b1d8775cb8" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126"
I20260811 22:14:42.634387 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83788aee4404493b59cc4b1d8775cb8" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72"
I20260811 22:14:42.634534 20612 raft_consensus.cc:3037] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.634534 20470 raft_consensus.cc:3037] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:42.635764 20612 raft_consensus.cc:2454] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 1.
I20260811 22:14:42.635764 20470 raft_consensus.cc:2454] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 1.
I20260811 22:14:42.636207 20240 leader_election.cc:302] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 381c4b8188a1477f9dc5c73c6a537126, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:42.636471 20709 raft_consensus.cc:2781] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:42.636734 20709 raft_consensus.cc:686] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 1 LEADER]: Becoming Leader. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Running, Role: LEADER
I20260811 22:14:42.637089 20709 consensus_queue.cc:237] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:42.639796 20138 catalog_manager.cc:5495] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1597bff28584e769fa081ca601c5b72 (127.17.198.1). New cstate: current_term: 1 leader_uuid: "e1597bff28584e769fa081ca601c5b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260811 22:14:42.660151 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.655991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.17.198.1:46367)
0811 22:14:42.656187 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.656191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.656205 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:42.656304 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.656306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.656378 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.656486 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.657034 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.657558 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.658555 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.658782 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.658882 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.659206 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.659717 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.659832 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.659961 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.660029 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 22:14:42.673384 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.669215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:49325 (local address 127.17.198.2:46347)
0811 22:14:42.669432 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.669438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.669657 (+ 219us) server_negotiation.cc:407] Connection header received
0811 22:14:42.669901 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.669910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.669967 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.670061 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.670371 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.670992 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.672021 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.672279 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.672408 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.672765 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.672931 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.673019 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.673142 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.673207 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:14:42.673403 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.669049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46347 (local address 127.17.198.1:49325)
0811 22:14:42.669555 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.669569 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.669720 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.670089 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.670092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.670100 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:42.670249 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.670255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.671141 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.671145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.671884 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.671894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.672417 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.672420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.672535 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.672589 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.673040 (+ 451us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.673047 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.673124 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.673217 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":175,"threads_started":1}
I20260811 22:14:42.680667 20612 raft_consensus.cc:1261] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Refusing update from remote peer e1597bff28584e769fa081ca601c5b72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.680665 20470 raft_consensus.cc:1261] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Refusing update from remote peer e1597bff28584e769fa081ca601c5b72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:42.681249 20709 consensus_queue.cc:1035] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.681552 20763 consensus_queue.cc:1035] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:42.701627 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.697692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:53349 (local address 127.17.198.1:46367)
0811 22:14:42.697828 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:42.697834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:42.697933 (+ 99us) server_negotiation.cc:407] Connection header received
0811 22:14:42.697993 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:42.697996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:42.698079 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:42.698158 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:42.698413 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.699158 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.700164 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.700478 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.700596 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.701089 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.701184 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.701269 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:42.701410 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 22:14:42.701464 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:14:42.701673 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:42.697587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46367 (local address 127.17.198.2:53349)
0811 22:14:42.697756 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:42.697774 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:42.697870 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:42.698180 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:42.698183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:42.698190 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:14:42.698292 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.698297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.699287 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.699290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.700010 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:42.700020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:42.700639 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:42.700643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:42.700778 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:42.700964 (+ 186us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:14:42.701308 (+ 344us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:14:42.701315 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:14:42.701396 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:14:42.701515 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:14:42.702626 20736 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:14:44.276073 20139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:44.276280 20139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:44.279325 20139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Sending DeleteTablet for 3 replicas of tablet b83788aee4404493b59cc4b1d8775cb8
I20260811 22:14:44.280155 20290 tablet_service.cc:1514] Processing DeleteTablet for tablet b83788aee4404493b59cc4b1d8775cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260811 22:14:44.280198 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet b83788aee4404493b59cc4b1d8775cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:59572
I20260811 22:14:44.280252 20448 tablet_service.cc:1514] Processing DeleteTablet for tablet b83788aee4404493b59cc4b1d8775cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:49416
I20260811 22:14:44.280618 20800 tablet_replica.cc:330] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: stopping tablet replica
I20260811 22:14:44.280618 20796 tablet_replica.cc:330] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: stopping tablet replica
I20260811 22:14:44.280869 20800 raft_consensus.cc:2229] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:44.280936 20796 raft_consensus.cc:2229] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:44.281052 20800 raft_consensus.cc:2258] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.281337 20796 raft_consensus.cc:2258] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.281983 20800 ts_tablet_manager.cc:1905] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:44.281983 20796 ts_tablet_manager.cc:1905] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:44.284786 20800 ts_tablet_manager.cc:1918] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:14:44.284786 20796 ts_tablet_manager.cc:1918] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:14:44.284911 20796 log.cc:1196] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/b83788aee4404493b59cc4b1d8775cb8
I20260811 22:14:44.284910 20800 log.cc:1196] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/b83788aee4404493b59cc4b1d8775cb8
I20260811 22:14:44.285224 20800 ts_tablet_manager.cc:1939] T b83788aee4404493b59cc4b1d8775cb8 P 1177ff94db254997b2a6915a54686a72: Deleting consensus metadata
I20260811 22:14:44.285250 20796 ts_tablet_manager.cc:1939] T b83788aee4404493b59cc4b1d8775cb8 P e1597bff28584e769fa081ca601c5b72: Deleting consensus metadata
I20260811 22:14:44.285647 20798 tablet_replica.cc:330] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: stopping tablet replica
I20260811 22:14:44.285879 20798 raft_consensus.cc:2229] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:44.286079 20798 raft_consensus.cc:2258] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.286330 20125 catalog_manager.cc:4841] TS e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367): tablet b83788aee4404493b59cc4b1d8775cb8 (table test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]) successfully deleted
I20260811 22:14:44.286423 20124 catalog_manager.cc:4841] TS 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347): tablet b83788aee4404493b59cc4b1d8775cb8 (table test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]) successfully deleted
I20260811 22:14:44.286849 20798 ts_tablet_manager.cc:1905] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:44.289500 20798 ts_tablet_manager.cc:1918] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:14:44.289616 20798 log.cc:1196] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/b83788aee4404493b59cc4b1d8775cb8
I20260811 22:14:44.289901 20798 ts_tablet_manager.cc:1939] T b83788aee4404493b59cc4b1d8775cb8 P 381c4b8188a1477f9dc5c73c6a537126: Deleting consensus metadata
I20260811 22:14:44.290829 20127 catalog_manager.cc:4841] TS 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005): tablet b83788aee4404493b59cc4b1d8775cb8 (table test-table [id=9b2135f7c73a428fbc2a9ee1c062b5c7]) successfully deleted
Aug 11 22:14:44 dist-test-slave-77v9 krb5kdc[20094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486484, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:14:44 dist-test-slave-77v9 krb5kdc[20094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486484, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:44.316509 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:44.306269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.17.198.62:40057)
0811 22:14:44.306592 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:44.306600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:44.306671 (+ 71us) server_negotiation.cc:407] Connection header received
0811 22:14:44.306846 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:44.306849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:44.306918 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:44.307060 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:44.308121 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:44.308691 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:44.309712 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:44.309955 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:44.310042 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:44.312905 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:44.312908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:44.312910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:44.312966 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:44.314662 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:44.315180 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:44.315184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:44.315186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:44.315285 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:44.315561 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:44.315563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:44.315565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:44.315797 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:44.315941 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:44.316199 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:14:44.316337 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":136,"threads_started":1}
I20260811 22:14:44.319799 20139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:44.320314 20139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:44.325917 20290 tablet_service.cc:1467] Processing CreateTablet for tablet c1da9fd6fb6e409590320511419af61e (DEFAULT_TABLE table=test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.326001 20603 tablet_service.cc:1467] Processing CreateTablet for tablet c1da9fd6fb6e409590320511419af61e (DEFAULT_TABLE table=test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.326040 20448 tablet_service.cc:1467] Processing CreateTablet for tablet c1da9fd6fb6e409590320511419af61e (DEFAULT_TABLE table=test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.326289 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1da9fd6fb6e409590320511419af61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.326308 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1da9fd6fb6e409590320511419af61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.326309 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1da9fd6fb6e409590320511419af61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.329764 20811 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Bootstrap starting.
I20260811 22:14:44.330207 20810 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Bootstrap starting.
I20260811 22:14:44.330823 20811 tablet_bootstrap.cc:654] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.331035 20809 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Bootstrap starting.
I20260811 22:14:44.331471 20810 tablet_bootstrap.cc:654] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.332048 20809 tablet_bootstrap.cc:654] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.332383 20811 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: No bootstrap required, opened a new log
I20260811 22:14:44.332528 20811 ts_tablet_manager.cc:1397] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:44.332891 20810 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: No bootstrap required, opened a new log
I20260811 22:14:44.332998 20810 ts_tablet_manager.cc:1397] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:44.333191 20811 raft_consensus.cc:348] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.333319 20811 raft_consensus.cc:374] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.333352 20811 raft_consensus.cc:729] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.333395 20810 raft_consensus.cc:348] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.333541 20810 raft_consensus.cc:374] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.333555 20809 tablet_bootstrap.cc:492] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: No bootstrap required, opened a new log
I20260811 22:14:44.333513 20811 consensus_queue.cc:260] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.333580 20810 raft_consensus.cc:729] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.333638 20809 ts_tablet_manager.cc:1397] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:44.333722 20810 consensus_queue.cc:260] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.333806 20811 ts_tablet_manager.cc:1428] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:44.334023 20810 ts_tablet_manager.cc:1428] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:44.334030 20809 raft_consensus.cc:348] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.334143 20809 raft_consensus.cc:374] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.334173 20809 raft_consensus.cc:729] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.334288 20809 consensus_queue.cc:260] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.334674 20809 ts_tablet_manager.cc:1428] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:44.467111 20786 raft_consensus.cc:482] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:44.467339 20786 raft_consensus.cc:504] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.467968 20786 leader_election.cc:288] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:44.468508 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1da9fd6fb6e409590320511419af61e" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72" is_pre_election: true
I20260811 22:14:44.468710 20470 raft_consensus.cc:2454] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 0.
I20260811 22:14:44.468710 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1da9fd6fb6e409590320511419af61e" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126" is_pre_election: true
I20260811 22:14:44.468890 20612 raft_consensus.cc:2454] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 0.
I20260811 22:14:44.469236 20236 leader_election.cc:302] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:44.469521 20786 raft_consensus.cc:2781] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:44.469610 20786 raft_consensus.cc:482] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:44.469671 20786 raft_consensus.cc:3037] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.470957 20786 raft_consensus.cc:504] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.471385 20786 leader_election.cc:288] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 election: Requested vote from peers 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347), 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005)
I20260811 22:14:44.472030 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1da9fd6fb6e409590320511419af61e" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72"
I20260811 22:14:44.472107 20612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1da9fd6fb6e409590320511419af61e" candidate_uuid: "e1597bff28584e769fa081ca601c5b72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381c4b8188a1477f9dc5c73c6a537126"
I20260811 22:14:44.472186 20470 raft_consensus.cc:3037] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.472272 20612 raft_consensus.cc:3037] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.473685 20470 raft_consensus.cc:2454] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 1.
I20260811 22:14:44.473919 20612 raft_consensus.cc:2454] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1597bff28584e769fa081ca601c5b72 in term 1.
I20260811 22:14:44.474196 20236 leader_election.cc:302] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1177ff94db254997b2a6915a54686a72, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:44.474507 20786 raft_consensus.cc:2781] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:44.474612 20786 raft_consensus.cc:686] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 1 LEADER]: Becoming Leader. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Running, Role: LEADER
I20260811 22:14:44.474766 20786 consensus_queue.cc:237] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.476709 20141 catalog_manager.cc:5495] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1597bff28584e769fa081ca601c5b72 (127.17.198.1). New cstate: current_term: 1 leader_uuid: "e1597bff28584e769fa081ca601c5b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260811 22:14:44.502099 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:44.498492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.17.198.1:46367)
0811 22:14:44.498820 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:44.498827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:44.498840 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:14:44.498890 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:44.498892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:44.498942 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:44.499021 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:44.499273 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:44.499823 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:44.500736 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:44.501153 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:44.501238 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:44.501602 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:44.501680 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:44.501738 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:44.501875 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:14:44.501928 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":133,"threads_started":1}
I20260811 22:14:44.510288 20470 raft_consensus.cc:1261] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Refusing update from remote peer e1597bff28584e769fa081ca601c5b72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:44.510304 20612 raft_consensus.cc:1261] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Refusing update from remote peer e1597bff28584e769fa081ca601c5b72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:44.510728 20786 consensus_queue.cc:1035] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:44.510891 20789 consensus_queue.cc:1035] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:44.549146 20141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:44.549337 20141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:44.551889 20141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Sending DeleteTablet for 3 replicas of tablet c1da9fd6fb6e409590320511419af61e
I20260811 22:14:44.552654 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet c1da9fd6fb6e409590320511419af61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:59572
I20260811 22:14:44.552654 20290 tablet_service.cc:1514] Processing DeleteTablet for tablet c1da9fd6fb6e409590320511419af61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260811 22:14:44.552655 20448 tablet_service.cc:1514] Processing DeleteTablet for tablet c1da9fd6fb6e409590320511419af61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:44 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:49416
I20260811 22:14:44.552896 20798 tablet_replica.cc:330] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: stopping tablet replica
I20260811 22:14:44.552906 20800 tablet_replica.cc:330] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: stopping tablet replica
I20260811 22:14:44.552956 20796 tablet_replica.cc:330] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: stopping tablet replica
I20260811 22:14:44.553030 20800 raft_consensus.cc:2229] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:44.553031 20798 raft_consensus.cc:2229] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:44.553067 20796 raft_consensus.cc:2229] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:44.553182 20800 raft_consensus.cc:2258] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.553226 20798 raft_consensus.cc:2258] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.553293 20796 raft_consensus.cc:2258] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:44.553591 20141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:14:44.554196 20798 ts_tablet_manager.cc:1905] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:14:44.554208 20141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:44.554435 20800 ts_tablet_manager.cc:1905] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:44.557377 20796 ts_tablet_manager.cc:1905] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:44.557380 20798 ts_tablet_manager.cc:1918] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:44.557399 20800 ts_tablet_manager.cc:1918] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:44.557477 20798 log.cc:1196] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/c1da9fd6fb6e409590320511419af61e
I20260811 22:14:44.557482 20800 log.cc:1196] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/c1da9fd6fb6e409590320511419af61e
I20260811 22:14:44.557938 20798 ts_tablet_manager.cc:1939] T c1da9fd6fb6e409590320511419af61e P 381c4b8188a1477f9dc5c73c6a537126: Deleting consensus metadata
I20260811 22:14:44.557934 20800 ts_tablet_manager.cc:1939] T c1da9fd6fb6e409590320511419af61e P 1177ff94db254997b2a6915a54686a72: Deleting consensus metadata
I20260811 22:14:44.558666 20124 catalog_manager.cc:4841] TS 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347): tablet c1da9fd6fb6e409590320511419af61e (table test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]) successfully deleted
I20260811 22:14:44.559618 20290 tablet_service.cc:1467] Processing CreateTablet for tablet cfcd526766a2465db75af01bad033e95 (DEFAULT_TABLE table=test-table [id=e222d7ab69fd4d96bd60d65e8cb5b5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.559878 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcd526766a2465db75af01bad033e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.560081 20796 ts_tablet_manager.cc:1918] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:14:44.560142 20796 log.cc:1196] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/c1da9fd6fb6e409590320511419af61e
I20260811 22:14:44.560338 20796 ts_tablet_manager.cc:1939] T c1da9fd6fb6e409590320511419af61e P e1597bff28584e769fa081ca601c5b72: Deleting consensus metadata
I20260811 22:14:44.561074 20127 catalog_manager.cc:4841] TS 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005): tablet c1da9fd6fb6e409590320511419af61e (table test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]) successfully deleted
I20260811 22:14:44.561112 20603 tablet_service.cc:1467] Processing CreateTablet for tablet cfcd526766a2465db75af01bad033e95 (DEFAULT_TABLE table=test-table [id=e222d7ab69fd4d96bd60d65e8cb5b5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.561394 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcd526766a2465db75af01bad033e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.562024 20811 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: Bootstrap starting.
I20260811 22:14:44.562093 20125 catalog_manager.cc:4841] TS e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367): tablet c1da9fd6fb6e409590320511419af61e (table test-table [id=27183d9ddb434b2c9c7e0c5557ec548e]) successfully deleted
I20260811 22:14:44.562242 20448 tablet_service.cc:1467] Processing CreateTablet for tablet cfcd526766a2465db75af01bad033e95 (DEFAULT_TABLE table=test-table [id=e222d7ab69fd4d96bd60d65e8cb5b5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:44.562518 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcd526766a2465db75af01bad033e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:44.562922 20811 tablet_bootstrap.cc:654] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.563560 20809 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Bootstrap starting.
I20260811 22:14:44.564384 20809 tablet_bootstrap.cc:654] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.564687 20810 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Bootstrap starting.
I20260811 22:14:44.565574 20811 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: No bootstrap required, opened a new log
I20260811 22:14:44.565645 20810 tablet_bootstrap.cc:654] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:44.565680 20811 ts_tablet_manager.cc:1397] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 22:14:44.566108 20809 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: No bootstrap required, opened a new log
I20260811 22:14:44.566213 20809 ts_tablet_manager.cc:1397] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:44.566653 20809 raft_consensus.cc:348] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.566690 20811 raft_consensus.cc:348] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.566783 20809 raft_consensus.cc:374] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.566818 20809 raft_consensus.cc:729] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.566834 20811 raft_consensus.cc:374] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.566874 20811 raft_consensus.cc:729] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1597bff28584e769fa081ca601c5b72, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.566896 20810 tablet_bootstrap.cc:492] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: No bootstrap required, opened a new log
I20260811 22:14:44.567005 20810 ts_tablet_manager.cc:1397] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:14:44.566990 20809 consensus_queue.cc:260] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.567020 20811 consensus_queue.cc:260] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.567296 20811 ts_tablet_manager.cc:1428] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:14:44.567296 20809 ts_tablet_manager.cc:1428] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:44.567518 20810 raft_consensus.cc:348] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.567608 20810 raft_consensus.cc:374] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:44.567634 20810 raft_consensus.cc:729] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1177ff94db254997b2a6915a54686a72, State: Initialized, Role: FOLLOWER
I20260811 22:14:44.567926 20810 consensus_queue.cc:260] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.568295 20810 ts_tablet_manager.cc:1428] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:14:44.969166 20815 raft_consensus.cc:482] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:44.969355 20815 raft_consensus.cc:504] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.969902 20815 leader_election.cc:288] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347)
I20260811 22:14:44.970520 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcd526766a2465db75af01bad033e95" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72" is_pre_election: true
I20260811 22:14:44.970508 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcd526766a2465db75af01bad033e95" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72" is_pre_election: true
I20260811 22:14:44.970665 20314 raft_consensus.cc:2454] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 0.
I20260811 22:14:44.970716 20470 raft_consensus.cc:2454] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 0.
I20260811 22:14:44.971094 20547 leader_election.cc:302] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 381c4b8188a1477f9dc5c73c6a537126, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:44.971329 20815 raft_consensus.cc:2781] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:44.971418 20815 raft_consensus.cc:482] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:44.971475 20815 raft_consensus.cc:3037] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.972563 20815 raft_consensus.cc:504] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.973034 20815 leader_election.cc:288] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 election: Requested vote from peers e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367), 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347)
I20260811 22:14:44.973492 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcd526766a2465db75af01bad033e95" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1177ff94db254997b2a6915a54686a72"
I20260811 22:14:44.973492 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcd526766a2465db75af01bad033e95" candidate_uuid: "381c4b8188a1477f9dc5c73c6a537126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1597bff28584e769fa081ca601c5b72"
I20260811 22:14:44.973651 20314 raft_consensus.cc:3037] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.973651 20470 raft_consensus.cc:3037] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:44.974932 20470 raft_consensus.cc:2454] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 1.
I20260811 22:14:44.974932 20314 raft_consensus.cc:2454] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 381c4b8188a1477f9dc5c73c6a537126 in term 1.
I20260811 22:14:44.975304 20547 leader_election.cc:302] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 381c4b8188a1477f9dc5c73c6a537126, e1597bff28584e769fa081ca601c5b72; no voters:
I20260811 22:14:44.975543 20815 raft_consensus.cc:2781] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:44.975817 20815 raft_consensus.cc:686] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 LEADER]: Becoming Leader. State: Replica: 381c4b8188a1477f9dc5c73c6a537126, State: Running, Role: LEADER
I20260811 22:14:44.975991 20815 consensus_queue.cc:237] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } }
I20260811 22:14:44.978006 20138 catalog_manager.cc:5495] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 reported cstate change: term changed from 0 to 1, leader changed from <none> to 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "381c4b8188a1477f9dc5c73c6a537126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "381c4b8188a1477f9dc5c73c6a537126" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260811 22:14:45.038622 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:45.034740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.17.198.3:38005)
0811 22:14:45.035106 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:45.035116 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:45.035142 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:14:45.035192 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:45.035195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:45.035242 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:45.035325 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:45.035580 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:45.036188 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:45.037285 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:45.037548 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:45.037641 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:45.038105 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:45.038213 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:45.038285 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:45.038425 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:14:45.038481 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":121,"threads_started":1}
I20260811 22:14:45.046939 20470 raft_consensus.cc:1261] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Refusing update from remote peer 381c4b8188a1477f9dc5c73c6a537126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:45.046965 20314 raft_consensus.cc:1261] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Refusing update from remote peer 381c4b8188a1477f9dc5c73c6a537126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:45.047475 20826 consensus_queue.cc:1035] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1597bff28584e769fa081ca601c5b72" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:45.047650 20815 consensus_queue.cc:1035] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1177ff94db254997b2a6915a54686a72" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:14:45.065872 20778 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:14:46.637769 20139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:46.638058 20139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:46.641192 20139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c98029a237b84a41988f7ff72ed32415: Sending DeleteTablet for 3 replicas of tablet cfcd526766a2465db75af01bad033e95
I20260811 22:14:46.642105 20290 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcd526766a2465db75af01bad033e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:46 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260811 22:14:46.642105 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcd526766a2465db75af01bad033e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:46 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:59572
I20260811 22:14:46.642143 20448 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcd526766a2465db75af01bad033e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:46 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:49416
I20260811 22:14:46.642555 20863 tablet_replica.cc:330] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: stopping tablet replica
I20260811 22:14:46.642562 20862 tablet_replica.cc:330] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72: stopping tablet replica
I20260811 22:14:46.642666 20864 tablet_replica.cc:330] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: stopping tablet replica
I20260811 22:14:46.642746 20862 raft_consensus.cc:2229] T cfcd526766a2465db75af01bad033e95 P e1597bff28584e769fa081ca601c5b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:46.642840 20864 raft_consensus.cc:2229] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:46.642951 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20219
I20260811 22:14:46.642818 20863 raft_consensus.cc:2229] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:46.642987 20864 raft_consensus.cc:2258] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:46.647308 20864 ts_tablet_manager.cc:1905] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:46.648739 20863 raft_consensus.cc:2258] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:46.650802 20863 ts_tablet_manager.cc:1905] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:46.652240 20864 ts_tablet_manager.cc:1918] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:14:46.652354 20864 log.cc:1196] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/cfcd526766a2465db75af01bad033e95
I20260811 22:14:46.652645 20864 ts_tablet_manager.cc:1939] T cfcd526766a2465db75af01bad033e95 P 1177ff94db254997b2a6915a54686a72: Deleting consensus metadata
I20260811 22:14:46.655229 20863 ts_tablet_manager.cc:1918] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:14:46.655345 20863 log.cc:1196] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/cfcd526766a2465db75af01bad033e95
I20260811 22:14:46.655751 20863 ts_tablet_manager.cc:1939] T cfcd526766a2465db75af01bad033e95 P 381c4b8188a1477f9dc5c73c6a537126: Deleting consensus metadata
I20260811 22:14:46.656414 20124 catalog_manager.cc:4841] TS 1177ff94db254997b2a6915a54686a72 (127.17.198.2:46347): tablet cfcd526766a2465db75af01bad033e95 (table test-table [id=e222d7ab69fd4d96bd60d65e8cb5b5b5]) successfully deleted
I20260811 22:14:46.656893 20127 catalog_manager.cc:4841] TS 381c4b8188a1477f9dc5c73c6a537126 (127.17.198.3:38005): tablet cfcd526766a2465db75af01bad033e95 (table test-table [id=e222d7ab69fd4d96bd60d65e8cb5b5b5]) successfully deleted
W20260811 22:14:46.662698 20125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:46367 (error 108)
I20260811 22:14:46.663041 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20373
I20260811 22:14:46.666960 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:46.665724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:46367 (local address 127.0.0.1:53670)
0811 22:14:46.666700 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:46.666806 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":117,"threads_started":1}
W20260811 22:14:46.667182 20125 catalog_manager.cc:4584] TS e1597bff28584e769fa081ca601c5b72 (127.17.198.1:46367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfcd526766a2465db75af01bad033e95: Network error: Client connection negotiation failed: client connection to 127.17.198.1:46367: connect: Connection refused (error 111)
I20260811 22:14:46.673717 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20527
I20260811 22:14:46.684062 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20107
2026-08-11T22:14:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7448 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:46 dist-test-slave-77v9 krb5kdc[20869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:46 dist-test-slave-77v9 krb5kdc[20869](info): set up 2 sockets
Aug 11 22:14:46 dist-test-slave-77v9 krb5kdc[20869](info): commencing operation
krb5kdc: starting...
W20260811 22:14:47.898130 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:47 dist-test-slave-77v9 krb5kdc[20869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:48.054352 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35449
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:48.178050 20882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:48.178427 20882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:48.178503 20882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:48.182924 20882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:48.183045 20882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:48.183081 20882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:48.183120 20882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:48.183161 20882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:48.189251 20882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:48.190565 20882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:48.191764 20882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:48.202235 20888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.202245 20892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.202270 20889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:48.202406 20882 server_base.cc:1034] running on GCE node
I20260811 22:14:48.203259 20882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:48.204619 20882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:48.205948 20882 hybrid_clock.cc:648] HybridClock initialized: now 1786486488205909 us; error 49 us; skew 500 ppm
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:48.209471 20882 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:48.210683 20882 webserver.cc:460] Webserver started at http://127.17.198.62:42951/ using document root <none> and password file <none>
I20260811 22:14:48.211422 20882 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:48.211544 20882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:48.211829 20882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:48.214399 20882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "bc9c7e8c00e14920b79d679af7b067e3"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "cb0f181c00dafb7a8b565948992def24"
server_key_iv: "f55bd578977f3ece61e4e311a667d648"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.215161 20882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "bc9c7e8c00e14920b79d679af7b067e3"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "cb0f181c00dafb7a8b565948992def24"
server_key_iv: "f55bd578977f3ece61e4e311a667d648"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.219835 20882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:48.223301 20899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.225232 20882 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:48.225455 20882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "bc9c7e8c00e14920b79d679af7b067e3"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "cb0f181c00dafb7a8b565948992def24"
server_key_iv: "f55bd578977f3ece61e4e311a667d648"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.225618 20882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:48.238277 20882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:48.242475 20882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:48.242744 20882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:48.251540 20882 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35449
I20260811 22:14:48.251569 20956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35449 every 8 connection(s)
I20260811 22:14:48.252739 20882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:48.253417 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20882
I20260811 22:14:48.253571 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:48.253842 18200 external_mini_cluster.cc:1428] Setting key e12532362af0d150a17c7362b307c50e
I20260811 22:14:48.256518 20957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:48.263130 20957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3: Bootstrap starting.
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:48.266131 20957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:48.267022 20957 log.cc:824] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:48.269305 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:48.255422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.17.198.62:35449)
0811 22:14:48.255921 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:48.255944 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:48.255981 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:14:48.256678 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:48.256702 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:48.257135 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:48.257502 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:48.258410 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.259451 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.260363 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.260958 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.261130 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:48.263905 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:48.263911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:48.263949 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:48.263985 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:48.265908 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.266520 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.266529 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.266540 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.266657 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.266968 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.266972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.266975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.267462 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:48.267675 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:48.268037 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 22:14:48.268386 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":154,"threads_started":1}
I20260811 22:14:48.269718 20957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3: No bootstrap required, opened a new log
I20260811 22:14:48.274199 20957 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } }
I20260811 22:14:48.274529 20957 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:48.274621 20957 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9c7e8c00e14920b79d679af7b067e3, State: Initialized, Role: FOLLOWER
I20260811 22:14:48.275113 20957 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } }
I20260811 22:14:48.275308 20957 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:48.275396 20957 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:48.275527 20957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:48.276667 20957 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } }
I20260811 22:14:48.277151 20957 leader_election.cc:302] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc9c7e8c00e14920b79d679af7b067e3; no voters:
I20260811 22:14:48.277570 20957 leader_election.cc:288] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:48.277743 20962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:48.278024 20962 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [term 1 LEADER]: Becoming Leader. State: Replica: bc9c7e8c00e14920b79d679af7b067e3, State: Running, Role: LEADER
I20260811 22:14:48.278402 20962 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } }
I20260811 22:14:48.278985 20957 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:48.280124 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9c7e8c00e14920b79d679af7b067e3. Latest consensus state: current_term: 1 leader_uuid: "bc9c7e8c00e14920b79d679af7b067e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } } }
I20260811 22:14:48.280263 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:48.280602 20963 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc9c7e8c00e14920b79d679af7b067e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9c7e8c00e14920b79d679af7b067e3" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35449 } } }
I20260811 22:14:48.280764 20963 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:48.280864 20971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:48.284396 20971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:48.291369 20971 catalog_manager.cc:1347] Generated new cluster ID: e6f4267ca2854fa1b3da68f6c2809dee
I20260811 22:14:48.291476 20971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:48.311398 20971 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:48.312736 20971 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:48.324335 20971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc9c7e8c00e14920b79d679af7b067e3: Generated new TSK 0
I20260811 22:14:48.325335 20971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:48.360777 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:48.483202 20985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:48.483505 20985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:48.483551 20985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:48.487887 20985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:48.487960 20985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:48.488025 20985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:48.493666 20985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:48.494829 20985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:48.495998 20985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:48.506872 20991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.506872 20993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.506937 20990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:48.507438 20985 server_base.cc:1034] running on GCE node
I20260811 22:14:48.508181 20985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:48.509110 20985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:48.510299 20985 hybrid_clock.cc:648] HybridClock initialized: now 1786486488510277 us; error 43 us; skew 500 ppm
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:48.513738 20985 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:48.514786 20985 webserver.cc:460] Webserver started at http://127.17.198.1:34253/ using document root <none> and password file <none>
I20260811 22:14:48.515416 20985 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:48.515493 20985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:48.515761 20985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:48.518249 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "0ca32908708a4e54a4d2dbf7e507baf4"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "179c9e809677c68cecd57dd73b575936"
server_key_iv: "047fb405ec62c6471c9d808eb09797bb"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.518877 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "0ca32908708a4e54a4d2dbf7e507baf4"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "179c9e809677c68cecd57dd73b575936"
server_key_iv: "047fb405ec62c6471c9d808eb09797bb"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.523370 20985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:48.526333 21001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.528010 20985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:48.528205 20985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "0ca32908708a4e54a4d2dbf7e507baf4"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "179c9e809677c68cecd57dd73b575936"
server_key_iv: "047fb405ec62c6471c9d808eb09797bb"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.528365 20985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:48.542907 20985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:48.546823 20985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:48.547132 20985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:48.547807 20985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:48.548734 20985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:48.548816 20985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.548964 20985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:48.549009 20985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.559119 20985 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:35569
I20260811 22:14:48.559173 21122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:35569 every 8 connection(s)
I20260811 22:14:48.560236 20985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:48.569442 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 20985
I20260811 22:14:48.569578 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:48.569906 18200 external_mini_cluster.cc:1428] Setting key 3db6b4aabc5deca6c6ff57fd117d731c
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:48.576243 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:48.562441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:47395 (local address 127.17.198.62:35449)
0811 22:14:48.562601 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:48.562605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:48.563568 (+ 963us) server_negotiation.cc:407] Connection header received
0811 22:14:48.565086 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:48.565089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:48.565142 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:48.565222 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:48.566632 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.567192 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.568595 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.568901 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.569000 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:48.572364 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:48.572367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:48.572370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:48.572393 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:48.573946 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.574547 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.574551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.574553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.574639 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.575031 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.575035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.575037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.575236 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:48.575347 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:48.575904 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:14:48.576066 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:14:48.577242 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:48.562812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35449 (local address 127.17.198.1:47395)
0811 22:14:48.563382 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:48.563433 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:48.564752 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:48.565433 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:48.565441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:48.565884 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:48.566444 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:48.566460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.567355 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.567359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:48.568453 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:48.568464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.569147 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.569154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:48.569397 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:48.570228 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:48.570250 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:48.572171 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:48.574091 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:48.574099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:48.574120 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:48.574429 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:48.574739 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:48.574744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:48.574746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:48.574910 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:48.575351 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:48.575361 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:48.575648 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 22:14:48.575863 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:14:48.576249 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":138,"threads_started":1}
I20260811 22:14:48.578706 21123 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35449
I20260811 22:14:48.578987 21123 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:48.579552 21123 heartbeater.cc:511] Master 127.17.198.62:35449 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:48.581601 20918 ts_manager.cc:194] Registered new tserver with Master: 0ca32908708a4e54a4d2dbf7e507baf4 (127.17.198.1:35569)
I20260811 22:14:48.582927 20918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:47395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:48.606107 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:48.731240 21129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:48.731546 21129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:48.731593 21129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:48.736011 21129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:48.736092 21129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:48.736157 21129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:48.741905 21129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:48.743027 21129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:48.744155 21129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:48.755041 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.755051 21135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:48.755067 21137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:48.755300 21129 server_base.cc:1034] running on GCE node
I20260811 22:14:48.755822 21129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:48.756570 21129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:48.757771 21129 hybrid_clock.cc:648] HybridClock initialized: now 1786486488757748 us; error 84 us; skew 500 ppm
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:48.761235 21129 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:48.762377 21129 webserver.cc:460] Webserver started at http://127.17.198.2:38309/ using document root <none> and password file <none>
I20260811 22:14:48.763057 21129 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:48.763177 21129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:48.763453 21129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:48.766075 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "68aa73c14824408c89c2db33292f415a"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "d216f53effa03e5464a5e520aba7a1b1"
server_key_iv: "3da606dff62d437c0bb57d46228d2794"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.766825 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "68aa73c14824408c89c2db33292f415a"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "d216f53effa03e5464a5e520aba7a1b1"
server_key_iv: "3da606dff62d437c0bb57d46228d2794"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.771394 21129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:14:48.774442 21144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.775981 21129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:48.776182 21129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "68aa73c14824408c89c2db33292f415a"
format_stamp: "Formatted at 2026-08-11 22:14:48 on dist-test-slave-77v9"
server_key: "d216f53effa03e5464a5e520aba7a1b1"
server_key_iv: "3da606dff62d437c0bb57d46228d2794"
server_key_version: "encryptionkey@0"
I20260811 22:14:48.776330 21129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:48.795329 21129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:48.798883 21129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:48.799120 21129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:48.799748 21129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:48.800637 21129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:48.800709 21129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:48.800781 21129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:48.800851 21129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:14:48.811020 21129 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:40837
I20260811 22:14:48.811071 21268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:40837 every 8 connection(s)
I20260811 22:14:48.812158 21129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:48.814802 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21129
I20260811 22:14:48.814930 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:48.815290 18200 external_mini_cluster.cc:1428] Setting key f83cdf14d58a147e4e8fcf0a818d8b9b
Aug 11 22:14:48 dist-test-slave-77v9 krb5kdc[20869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:14:48.830191 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:48.814465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:48189 (local address 127.17.198.62:35449)
0811 22:14:48.814660 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:48.814667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:48.815671 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 22:14:48.816987 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:48.816992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:48.817092 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:48.817213 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:48.818964 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.819568 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.821411 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.821800 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.821956 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:48.825500 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:48.825504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:48.825506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:48.825536 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:48.827295 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.828074 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.828078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.828080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.828162 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:48.828624 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:48.828627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:48.828629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:48.828915 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:48.829018 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:48.829822 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 22:14:48.829983 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:14:48.831173 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:48.814812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35449 (local address 127.17.198.2:48189)
0811 22:14:48.815390 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:48.815448 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:48.816705 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:48.817473 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:48.817483 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:48.817900 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:48.818673 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:48.818698 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.819728 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.819734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:48.821221 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:48.821232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:48.822036 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:48.822040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:48.822212 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:48.822967 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:48.822985 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:48.825196 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:48.827456 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:48.827467 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:48.827492 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:48.827944 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:48.828257 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:48.828260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:48.828262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:48.828508 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:48.829076 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:48.829083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:48.829407 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 22:14:48.829750 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 22:14:48.830162 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":183,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:14:48.832567 21269 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35449
I20260811 22:14:48.832890 21269 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:48.833468 21269 heartbeater.cc:511] Master 127.17.198.62:35449 requested a full tablet report, sending...
I20260811 22:14:48.834666 20918 ts_manager.cc:194] Registered new tserver with Master: 68aa73c14824408c89c2db33292f415a (127.17.198.2:40837)
I20260811 22:14:48.835279 20918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:48189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:48.854130 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:48.977264 21274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:48.977586 21274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:48.977679 21274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:48.982467 21274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:48.982559 21274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:48.982666 21274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:48.988474 21274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:48.989791 21274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:48.990937 21274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:49.001762 21280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:49.001870 21274 server_base.cc:1034] running on GCE node
W20260811 22:14:49.001994 21279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:49.001768 21282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:49.002666 21274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:49.003443 21274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:49.004670 21274 hybrid_clock.cc:648] HybridClock initialized: now 1786486489004633 us; error 47 us; skew 500 ppm
Aug 11 22:14:49 dist-test-slave-77v9 krb5kdc[20869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:49.008227 21274 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:49.009388 21274 webserver.cc:460] Webserver started at http://127.17.198.3:44341/ using document root <none> and password file <none>
I20260811 22:14:49.010032 21274 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:49.010106 21274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:49.010362 21274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:49.013091 21274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "31d7ff35f9894c8b896dad2032729b16"
format_stamp: "Formatted at 2026-08-11 22:14:49 on dist-test-slave-77v9"
server_key: "f2c441ecc0eb0594d00ab01b5f6e84c1"
server_key_iv: "1a4a5dc04225aa8ee14d52b238a667f1"
server_key_version: "encryptionkey@0"
I20260811 22:14:49.013716 21274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "31d7ff35f9894c8b896dad2032729b16"
format_stamp: "Formatted at 2026-08-11 22:14:49 on dist-test-slave-77v9"
server_key: "f2c441ecc0eb0594d00ab01b5f6e84c1"
server_key_iv: "1a4a5dc04225aa8ee14d52b238a667f1"
server_key_version: "encryptionkey@0"
I20260811 22:14:49.018467 21274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:49.021785 21289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:49.023478 21274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:14:49.023665 21274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "31d7ff35f9894c8b896dad2032729b16"
format_stamp: "Formatted at 2026-08-11 22:14:49 on dist-test-slave-77v9"
server_key: "f2c441ecc0eb0594d00ab01b5f6e84c1"
server_key_iv: "1a4a5dc04225aa8ee14d52b238a667f1"
server_key_version: "encryptionkey@0"
I20260811 22:14:49.023811 21274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:49.056401 21274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:49.060755 21274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:49.061040 21274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:49.061758 21274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:49.062700 21274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:49.062750 21274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:49.062821 21274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:49.062856 21274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:49.074501 21274 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:44805
I20260811 22:14:49.074533 21414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:44805 every 8 connection(s)
I20260811 22:14:49.075618 21274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:49.083690 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21274
I20260811 22:14:49.083806 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:49.084137 18200 external_mini_cluster.cc:1428] Setting key d8ee6bc6eac12fbefa209a317544aeeb
Aug 11 22:14:49 dist-test-slave-77v9 krb5kdc[20869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:49.092748 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:49.077793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:36083 (local address 127.17.198.62:35449)
0811 22:14:49.078050 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:49.078055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:49.079046 (+ 991us) server_negotiation.cc:407] Connection header received
0811 22:14:49.080678 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:49.080684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:49.080830 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:49.080959 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:49.082498 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:49.083094 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:49.084483 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:49.084760 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:49.084991 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:49.088371 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:49.088375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:49.088378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:49.088406 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:49.090369 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:49.091014 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:49.091018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:49.091019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:49.091118 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:49.091524 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:49.091527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:49.091529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:49.091744 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:49.091826 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:49.092491 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 22:14:49.092616 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 22:14:49.093761 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:49.078283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35449 (local address 127.17.198.3:36083)
0811 22:14:49.078816 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:49.078902 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:49.080348 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:49.081170 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:49.081177 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:49.081657 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:49.082290 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:49.082306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:49.083220 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:49.083223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:49.084323 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:49.084335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:49.085045 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:49.085050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:49.085255 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:49.086266 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:49.086286 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:49.088153 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:49.090510 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:49.090517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:49.090533 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:49.090857 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:49.091240 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:49.091243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:49.091244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:49.091410 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:49.091857 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:49.091863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:49.092213 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 22:14:49.092404 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:14:49.092728 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":141,"threads_started":1}
I20260811 22:14:49.095185 21415 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35449
I20260811 22:14:49.095494 21415 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:49.096095 21415 heartbeater.cc:511] Master 127.17.198.62:35449 requested a full tablet report, sending...
I20260811 22:14:49.097424 20918 ts_manager.cc:194] Registered new tserver with Master: 31d7ff35f9894c8b896dad2032729b16 (127.17.198.3:44805)
I20260811 22:14:49.098191 20918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:36083
I20260811 22:14:49.099527 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:14:49.116487 20960 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:49.114881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.17.198.62:35449)
0811 22:14:49.115037 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:49.115041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:49.115125 (+ 84us) server_negotiation.cc:407] Connection header received
0811 22:14:49.115253 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:49.115256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:49.115309 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:49.115401 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:49.116345 (+ 944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48008: BlockingRecv error: recv got EOF from 127.0.0.1:48008 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:14:49.118671 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20985
I20260811 22:14:49.128677 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21129
I20260811 22:14:49.140416 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21274
I20260811 22:14:49.149480 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 20882
2026-08-11T22:14:49Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2462 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:49 dist-test-slave-77v9 krb5kdc[21429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:49 dist-test-slave-77v9 krb5kdc[21429](info): set up 2 sockets
Aug 11 22:14:49 dist-test-slave-77v9 krb5kdc[21429](info): commencing operation
krb5kdc: starting...
W20260811 22:14:50.342542 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:50.494074 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35925
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:50.622705 21442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:50.623100 21442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:50.623183 21442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:50.628883 21442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:50.629036 21442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:50.629081 21442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:50.629124 21442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:50.629164 21442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:50.635104 21442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:50.636409 21442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:50.637603 21442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:50.647512 21449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:50.647518 21452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:50.647653 21442 server_base.cc:1034] running on GCE node
W20260811 22:14:50.647506 21450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:50.648682 21442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:50.650038 21442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:50.651234 21442 hybrid_clock.cc:648] HybridClock initialized: now 1786486490651204 us; error 59 us; skew 500 ppm
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486490, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:50.654682 21442 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:50.655694 21442 webserver.cc:460] Webserver started at http://127.17.198.62:35633/ using document root <none> and password file <none>
I20260811 22:14:50.656267 21442 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:50.656334 21442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:50.656522 21442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:50.658865 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "bf7d0680cba94464819007073a889429"
format_stamp: "Formatted at 2026-08-11 22:14:50 on dist-test-slave-77v9"
server_key: "0715d4cc9d4c50a8eb852c9ac96a8208"
server_key_iv: "dd36b98307aebce085655df95171fbf6"
server_key_version: "encryptionkey@0"
I20260811 22:14:50.659467 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "bf7d0680cba94464819007073a889429"
format_stamp: "Formatted at 2026-08-11 22:14:50 on dist-test-slave-77v9"
server_key: "0715d4cc9d4c50a8eb852c9ac96a8208"
server_key_iv: "dd36b98307aebce085655df95171fbf6"
server_key_version: "encryptionkey@0"
I20260811 22:14:50.663800 21442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:50.666793 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:50.668444 21442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:50.668637 21442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "bf7d0680cba94464819007073a889429"
format_stamp: "Formatted at 2026-08-11 22:14:50 on dist-test-slave-77v9"
server_key: "0715d4cc9d4c50a8eb852c9ac96a8208"
server_key_iv: "dd36b98307aebce085655df95171fbf6"
server_key_version: "encryptionkey@0"
I20260811 22:14:50.668797 21442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:50.688030 21442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:50.692435 21442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:50.692711 21442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:50.700994 21442 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35925
I20260811 22:14:50.701463 21519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35925 every 8 connection(s)
I20260811 22:14:50.702193 21442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:50.703397 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21442
I20260811 22:14:50.703505 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:50.703830 18200 external_mini_cluster.cc:1428] Setting key 2d3ffee6b7667a82c1af06b0e340a822
I20260811 22:14:50.706077 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:50.712796 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429: Bootstrap starting.
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:50.715950 21520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:50.716894 21520 log.cc:824] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:50.719668 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429: No bootstrap required, opened a new log
I20260811 22:14:50.720360 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:50.705424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.17.198.62:35925)
0811 22:14:50.705988 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:50.706005 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:50.706044 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:14:50.706810 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:50.706830 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:50.707216 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:50.707573 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:50.708496 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:50.709594 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:50.710512 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:50.711015 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:50.711198 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:50.714188 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:50.714194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:50.714208 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:50.714244 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:50.716538 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:50.717246 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:50.717252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:50.717257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:50.717438 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:50.717823 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:50.717827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:50.717829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:50.718366 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:50.718544 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:50.719017 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 22:14:50.719404 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":179,"threads_started":1}
I20260811 22:14:50.724305 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:50.724699 21520 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } }
I20260811 22:14:50.725100 21520 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:50.725180 21520 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7d0680cba94464819007073a889429, State: Initialized, Role: FOLLOWER
I20260811 22:14:50.725742 21520 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } }
I20260811 22:14:50.725966 21520 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:50.726053 21520 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:50.726184 21520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:50.727722 21520 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } }
I20260811 22:14:50.728179 21520 leader_election.cc:302] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf7d0680cba94464819007073a889429; no voters:
I20260811 22:14:50.728627 21520 leader_election.cc:288] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:50.728739 21527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:50.728995 21527 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [term 1 LEADER]: Becoming Leader. State: Replica: bf7d0680cba94464819007073a889429, State: Running, Role: LEADER
I20260811 22:14:50.729436 21527 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } }
I20260811 22:14:50.730248 21520 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:50.730831 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf7d0680cba94464819007073a889429. Latest consensus state: current_term: 1 leader_uuid: "bf7d0680cba94464819007073a889429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } } }
I20260811 22:14:50.731604 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:50.732203 21536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:50.733307 21528 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf7d0680cba94464819007073a889429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7d0680cba94464819007073a889429" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35925 } } }
I20260811 22:14:50.733810 21528 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:50.736196 21536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:50.743825 21536 catalog_manager.cc:1347] Generated new cluster ID: 5fcd2b8ad7b54354a9b41c1d3575f815
I20260811 22:14:50.743997 21536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:50.749325 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:50.742610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.17.198.62:35925)
0811 22:14:50.742730 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:50.742735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:50.743226 (+ 491us) server_negotiation.cc:407] Connection header received
0811 22:14:50.743410 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:50.743413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:50.743468 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:50.743568 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:50.744344 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:50.745168 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:50.746475 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:50.746849 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:50.746949 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:50.748086 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:50.748090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:50.748093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:50.748726 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:50.748795 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:50.748846 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:50.748936 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:50.749082 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:14:50.749161 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:14:50.769398 21536 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:50.770789 21536 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:50.778784 21536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf7d0680cba94464819007073a889429: Generated new TSK 0
I20260811 22:14:50.779641 21536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:50.793835 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21442
2026-08-11T22:14:50Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1641 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21555](info): set up 2 sockets
Aug 11 22:14:50 dist-test-slave-77v9 krb5kdc[21555](info): commencing operation
krb5kdc: starting...
W20260811 22:14:51.948050 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:52.101584 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33833
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:33833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:52.223438 21570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:52.223780 21570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:52.223826 21570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:52.228186 21570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:52.228286 21570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:52.228307 21570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:52.228322 21570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:52.228339 21570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:52.234283 21570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:33833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:52.235476 21570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:52.236569 21570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:52.246136 21576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:52.246130 21577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.246245 21570 server_base.cc:1034] running on GCE node
W20260811 22:14:52.246133 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.247134 21570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:52.248463 21570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:52.249683 21570 hybrid_clock.cc:648] HybridClock initialized: now 1786486492249662 us; error 35 us; skew 500 ppm
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:52.257246 21570 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:52.258584 21570 webserver.cc:460] Webserver started at http://127.17.198.62:41141/ using document root <none> and password file <none>
I20260811 22:14:52.259321 21570 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:52.259456 21570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:52.259763 21570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:52.262368 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "0e490c369a6b4beb954beaf23e0b3521"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "0de1cfc0f89b0a5984f42e08e5559802"
server_key_iv: "3a1da3236c1ce3a74cbea7e08ccf49b8"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.263160 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "0e490c369a6b4beb954beaf23e0b3521"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "0de1cfc0f89b0a5984f42e08e5559802"
server_key_iv: "3a1da3236c1ce3a74cbea7e08ccf49b8"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.267990 21570 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:52.271325 21588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.272985 21570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:14:52.273165 21570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "0e490c369a6b4beb954beaf23e0b3521"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "0de1cfc0f89b0a5984f42e08e5559802"
server_key_iv: "3a1da3236c1ce3a74cbea7e08ccf49b8"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.273314 21570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:52.301023 21570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:52.304459 21570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:52.304702 21570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:52.312474 21570 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:33833
I20260811 22:14:52.312929 21644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:33833 every 8 connection(s)
I20260811 22:14:52.313658 21570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:52.316872 21645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:52.320849 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21570
I20260811 22:14:52.320990 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:52.321310 18200 external_mini_cluster.cc:1428] Setting key 27cbe5ead2b12073aede0422cf7fb228
I20260811 22:14:52.323567 21645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: Bootstrap starting.
I20260811 22:14:52.326385 21645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:52.327273 21645 log.cc:824] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:52.330346 21645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: No bootstrap required, opened a new log
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:52.334908 21645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } }
I20260811 22:14:52.335325 21645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:52.335455 21645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e490c369a6b4beb954beaf23e0b3521, State: Initialized, Role: FOLLOWER
I20260811 22:14:52.336025 21645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } }
I20260811 22:14:52.336205 21645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:52.336308 21645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:52.336422 21645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:52.337821 21645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } }
I20260811 22:14:52.338217 21645 leader_election.cc:302] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e490c369a6b4beb954beaf23e0b3521; no voters:
I20260811 22:14:52.338389 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:52.322909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.17.198.62:33833)
0811 22:14:52.323373 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:52.323390 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:52.323427 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:14:52.324153 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:52.324173 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:52.324528 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:52.324946 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:52.325951 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.327102 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.328235 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.328952 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.329096 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:52.332269 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:52.332275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:52.332292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:52.332325 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:52.334289 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.334947 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.334956 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.334963 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.335188 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.335607 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.335610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.335612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.336194 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:52.336429 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:52.336831 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:14:52.337170 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":162,"threads_started":1}
I20260811 22:14:52.338601 21645 leader_election.cc:288] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:52.338868 21650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:52.339222 21650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [term 1 LEADER]: Becoming Leader. State: Replica: 0e490c369a6b4beb954beaf23e0b3521, State: Running, Role: LEADER
I20260811 22:14:52.339563 21650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } }
I20260811 22:14:52.340111 21645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:52.342226 21652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e490c369a6b4beb954beaf23e0b3521. Latest consensus state: current_term: 1 leader_uuid: "0e490c369a6b4beb954beaf23e0b3521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } } }
I20260811 22:14:52.342382 21652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:52.343078 21651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e490c369a6b4beb954beaf23e0b3521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e490c369a6b4beb954beaf23e0b3521" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33833 } } }
I20260811 22:14:52.343191 21651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521 [sys.catalog]: This master's current role is: LEADER
W20260811 22:14:52.345580 21664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:52.345687 21664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:52.345772 21665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:52.346611 21665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:52.352566 21665 catalog_manager.cc:1347] Generated new cluster ID: 56e0da71f6204f608b14c3445b9f580f
I20260811 22:14:52.352662 21665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:52.367990 21665 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:52.369256 21665 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:52.376272 21665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e490c369a6b4beb954beaf23e0b3521: Generated new TSK 0
I20260811 22:14:52.377230 21665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:52.417130 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:52.540051 21672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:52.540434 21672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:52.540522 21672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:52.545152 21672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:52.545264 21672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:52.545363 21672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:52.551008 21672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:52.552361 21672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:52.553491 21672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:52.564168 21677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:52.564214 21679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.564267 21672 server_base.cc:1034] running on GCE node
W20260811 22:14:52.564214 21681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.565042 21672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:52.565774 21672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:52.566988 21672 hybrid_clock.cc:648] HybridClock initialized: now 1786486492566956 us; error 44 us; skew 500 ppm
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:52.570278 21672 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:52.571396 21672 webserver.cc:460] Webserver started at http://127.17.198.1:33905/ using document root <none> and password file <none>
I20260811 22:14:52.571969 21672 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:52.572032 21672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:52.572214 21672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:52.574434 21672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "3b7943e84b9d4778b9ff31d3b23a729a"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "25609c3539953c37adfd6a0a301ba9fe"
server_key_iv: "071416f8c77b3aa5152afb37b6007b7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.575053 21672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "3b7943e84b9d4778b9ff31d3b23a729a"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "25609c3539953c37adfd6a0a301ba9fe"
server_key_iv: "071416f8c77b3aa5152afb37b6007b7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.579679 21672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:14:52.582429 21688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.583956 21672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:52.584108 21672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "3b7943e84b9d4778b9ff31d3b23a729a"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "25609c3539953c37adfd6a0a301ba9fe"
server_key_iv: "071416f8c77b3aa5152afb37b6007b7e"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.584233 21672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:52.612370 21672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:52.616191 21672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:52.616398 21672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:52.617101 21672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:52.617990 21672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:52.618044 21672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.618088 21672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:52.618103 21672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.627563 21672 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:42429
I20260811 22:14:52.627583 21810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:42429 every 8 connection(s)
I20260811 22:14:52.628535 21672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:52.636291 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21672
I20260811 22:14:52.636461 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:52.636793 18200 external_mini_cluster.cc:1428] Setting key 0f4ab61f13bf161d87d740201a3183d4
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:52.646901 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:52.630437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:47601 (local address 127.17.198.62:33833)
0811 22:14:52.630598 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:52.630603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:52.631790 (+ 1187us) server_negotiation.cc:407] Connection header received
0811 22:14:52.633900 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:52.633904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:52.633981 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:52.634086 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:52.635453 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.636083 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.637887 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.638223 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.638377 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:52.641992 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:52.641996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:52.641998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:52.642049 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:52.644215 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.644949 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.644958 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.644959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.645046 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.645450 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.645453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.645455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.645665 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:52.645787 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:52.646524 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 22:14:52.646704 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:14:52.647907 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:52.630962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:33833 (local address 127.17.198.1:47601)
0811 22:14:52.631594 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:52.631656 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:52.633583 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:52.634260 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:52.634268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:52.634667 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:52.635222 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:52.635238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.636261 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.636267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:52.637735 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:52.637754 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.638452 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.638456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:52.638643 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:52.639351 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:52.639376 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:52.641699 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:52.644418 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:52.644426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:52.644450 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:52.644793 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:52.645149 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:52.645153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:52.645155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:52.645331 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:52.645789 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:52.645800 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:52.646122 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 22:14:52.646329 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:14:52.646901 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":167,"threads_started":1}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:52.649680 21811 heartbeater.cc:347] Connected to a master server at 127.17.198.62:33833
I20260811 22:14:52.649961 21811 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:52.650574 21811 heartbeater.cc:511] Master 127.17.198.62:33833 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:14:52.652477 21608 ts_manager.cc:194] Registered new tserver with Master: 3b7943e84b9d4778b9ff31d3b23a729a (127.17.198.1:42429)
I20260811 22:14:52.653775 21608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:47601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:52.673533 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:52.807256 21816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:52.807670 21816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:52.807761 21816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:52.812460 21816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:52.812572 21816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:52.812670 21816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:52.818451 21816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:52.819772 21816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:52.820983 21816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:52.831845 21822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:52.831880 21823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.832072 21816 server_base.cc:1034] running on GCE node
W20260811 22:14:52.831843 21825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:52.832664 21816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:52.833460 21816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:52.834661 21816 hybrid_clock.cc:648] HybridClock initialized: now 1786486492834644 us; error 32 us; skew 500 ppm
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:52.837982 21816 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:52.838956 21816 webserver.cc:460] Webserver started at http://127.17.198.2:38813/ using document root <none> and password file <none>
I20260811 22:14:52.839545 21816 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:52.839612 21816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:52.839792 21816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:52.842367 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "84f0d0f1ce3a458fbe9256085a4168d7"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "cf716e74180f407a941e7bcba15567ac"
server_key_iv: "98b33856a6fb3200495739c74759d466"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.843076 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "84f0d0f1ce3a458fbe9256085a4168d7"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "cf716e74180f407a941e7bcba15567ac"
server_key_iv: "98b33856a6fb3200495739c74759d466"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.847261 21816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:52.850212 21833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.851578 21816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:52.851747 21816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "84f0d0f1ce3a458fbe9256085a4168d7"
format_stamp: "Formatted at 2026-08-11 22:14:52 on dist-test-slave-77v9"
server_key: "cf716e74180f407a941e7bcba15567ac"
server_key_iv: "98b33856a6fb3200495739c74759d466"
server_key_version: "encryptionkey@0"
I20260811 22:14:52.851897 21816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:52.861509 21816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:52.865592 21816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:52.865828 21816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:52.866458 21816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:52.867389 21816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:52.867465 21816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.867537 21816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:52.867587 21816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:52.877585 21816 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:39109
I20260811 22:14:52.877643 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:39109 every 8 connection(s)
I20260811 22:14:52.878732 21816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:52.883128 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21816
I20260811 22:14:52.883296 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:52.883610 18200 external_mini_cluster.cc:1428] Setting key e55b445e32256a50be3451e18b7f4d86
Aug 11 22:14:52 dist-test-slave-77v9 krb5kdc[21555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486492, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:14:52.896835 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:52.881061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:47229 (local address 127.17.198.62:33833)
0811 22:14:52.881205 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:52.881214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:52.882260 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:14:52.884128 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:52.884132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:52.884194 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:52.884296 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:52.885897 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.886515 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.888214 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.888532 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.888646 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:52.891850 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:52.891855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:52.891857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:52.891888 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:52.894005 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.894846 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.894849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.894851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.894945 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:52.895367 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:52.895378 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:52.895380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:52.895652 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:52.895785 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:52.896434 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 22:14:52.896597 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:14:52.897817 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:52.881458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:33833 (local address 127.17.198.2:47229)
0811 22:14:52.882067 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:52.882122 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:52.883853 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:52.884518 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:52.884526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:52.885023 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:52.885672 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:52.885694 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.886669 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.886675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:52.888068 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:52.888079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:52.888710 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:52.888714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:52.888966 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:52.889782 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:52.889804 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:52.891604 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:52.894289 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:52.894297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:52.894318 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:52.894668 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:52.895058 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:52.895061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:52.895064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:52.895238 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:52.895776 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:52.895784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:52.896097 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:14:52.896331 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:14:52.896751 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:14:52.899313 21961 heartbeater.cc:347] Connected to a master server at 127.17.198.62:33833
I20260811 22:14:52.899608 21961 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:52.900363 21961 heartbeater.cc:511] Master 127.17.198.62:33833 requested a full tablet report, sending...
I20260811 22:14:52.901845 21608 ts_manager.cc:194] Registered new tserver with Master: 84f0d0f1ce3a458fbe9256085a4168d7 (127.17.198.2:39109)
I20260811 22:14:52.902565 21608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:47229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:52.922238 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:53.041014 21969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:53.041415 21969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:53.041507 21969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:53.045954 21969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:53.046092 21969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:53.046195 21969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:53.051980 21969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:33833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:53.053284 21969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:53.054373 21969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:14:53.065433 21969 server_base.cc:1034] running on GCE node
W20260811 22:14:53.065387 21974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:53.065389 21975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:53.065400 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:53.066185 21969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:53.067028 21969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:53.068398 21969 hybrid_clock.cc:648] HybridClock initialized: now 1786486493068352 us; error 61 us; skew 500 ppm
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486493, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:53.071843 21969 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:53.072917 21969 webserver.cc:460] Webserver started at http://127.17.198.3:42601/ using document root <none> and password file <none>
I20260811 22:14:53.073489 21969 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:53.073556 21969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:53.073732 21969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:53.076017 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "6dd325f8f07f4576bd4ef95e58a7d5e2"
format_stamp: "Formatted at 2026-08-11 22:14:53 on dist-test-slave-77v9"
server_key: "841a90a0242a4242013f4697af0ca806"
server_key_iv: "34dcc44ed3d69ee7effd44aeec6503ee"
server_key_version: "encryptionkey@0"
I20260811 22:14:53.076596 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "6dd325f8f07f4576bd4ef95e58a7d5e2"
format_stamp: "Formatted at 2026-08-11 22:14:53 on dist-test-slave-77v9"
server_key: "841a90a0242a4242013f4697af0ca806"
server_key_iv: "34dcc44ed3d69ee7effd44aeec6503ee"
server_key_version: "encryptionkey@0"
I20260811 22:14:53.081380 21969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:14:53.084188 21984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:53.085752 21969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:53.085945 21969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "6dd325f8f07f4576bd4ef95e58a7d5e2"
format_stamp: "Formatted at 2026-08-11 22:14:53 on dist-test-slave-77v9"
server_key: "841a90a0242a4242013f4697af0ca806"
server_key_iv: "34dcc44ed3d69ee7effd44aeec6503ee"
server_key_version: "encryptionkey@0"
I20260811 22:14:53.086066 21969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:53.121631 21969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:53.125468 21969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:53.125656 21969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:53.126437 21969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:53.127396 21969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:53.127452 21969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:53.127494 21969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:53.127509 21969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:53.137472 21969 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:34067
I20260811 22:14:53.137506 22109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:34067 every 8 connection(s)
I20260811 22:14:53.138455 21969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:53.141718 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 21969
I20260811 22:14:53.141924 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:53.142303 18200 external_mini_cluster.cc:1428] Setting key ae30ba8a0e0068682b156cbd8526822c
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[21555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486493, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:53.154490 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:53.140590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:51903 (local address 127.17.198.62:33833)
0811 22:14:53.140785 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:53.140789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:53.141876 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 22:14:53.143636 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:53.143640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:53.143706 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:53.143788 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:53.145227 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.145760 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.146966 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.147181 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.147261 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:53.150192 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:53.150196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:53.150199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:53.150230 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:53.151990 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:53.152642 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:53.152645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:53.152646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:53.152737 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:53.153294 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:53.153297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:53.153298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:53.153524 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:53.153611 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:53.154196 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 22:14:53.154319 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:14:53.155673 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:53.141091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:33833 (local address 127.17.198.3:51903)
0811 22:14:53.141653 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:53.141721 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:53.143389 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:53.143994 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:53.144001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:53.144414 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:53.145086 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:53.145101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.145894 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.145897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:53.146833 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:53.146843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.147328 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.147331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:53.147506 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:53.148120 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:53.148137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:53.149950 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:53.152158 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:53.152164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:53.152180 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:53.152515 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:53.152971 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:53.152974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:53.152976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:53.153163 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:53.153629 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:53.153634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:53.153928 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 22:14:53.154106 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:14:53.154455 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":147,"threads_started":1}
I20260811 22:14:53.157137 22110 heartbeater.cc:347] Connected to a master server at 127.17.198.62:33833
I20260811 22:14:53.157455 22110 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:53.158006 22110 heartbeater.cc:511] Master 127.17.198.62:33833 requested a full tablet report, sending...
I20260811 22:14:53.159142 21608 ts_manager.cc:194] Registered new tserver with Master: 6dd325f8f07f4576bd4ef95e58a7d5e2 (127.17.198.3:34067)
I20260811 22:14:53.159786 21608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:51903
I20260811 22:14:53.170231 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[21555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486493, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[21555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486493, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:53.201680 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:53.191058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.17.198.62:33833)
0811 22:14:53.191296 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:53.191303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:53.191496 (+ 193us) server_negotiation.cc:407] Connection header received
0811 22:14:53.191712 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:53.191715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:53.191764 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:53.191880 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:53.192766 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.193428 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.194418 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:53.194671 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:53.194814 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:53.197848 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:53.197852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:53.197854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:53.197888 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:53.199527 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:53.200090 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:53.200100 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:53.200101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:53.200195 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:53.200515 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:53.200519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:53.200521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:53.200799 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:53.201031 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:53.201338 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:14:53.201484 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 22:14:53.202302 21608 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40420
I20260811 22:14:53.203485 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21672
I20260811 22:14:53.214624 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21816
I20260811 22:14:53.223724 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21969
I20260811 22:14:53.237429 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 21570
2026-08-11T22:14:53Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2447 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[22122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[22122](info): set up 2 sockets
Aug 11 22:14:53 dist-test-slave-77v9 krb5kdc[22122](info): commencing operation
krb5kdc: starting...
W20260811 22:14:54.401089 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:54 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:54.553398 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43999
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:43999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:54.676990 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:54.677322 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:54.677367 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:54.681684 22135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:54.681792 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:54.681813 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:54.681831 22135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:54.681846 22135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:54.687739 22135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:43999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:54.689004 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:54.690065 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:54.699560 22140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:54.699560 22144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:54.699608 22135 server_base.cc:1034] running on GCE node
W20260811 22:14:54.699563 22141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:54.700450 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:54.701804 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:54.703014 22135 hybrid_clock.cc:648] HybridClock initialized: now 1786486494703003 us; error 35 us; skew 500 ppm
Aug 11 22:14:54 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:54.706185 22135 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:54.707290 22135 webserver.cc:460] Webserver started at http://127.17.198.62:43573/ using document root <none> and password file <none>
I20260811 22:14:54.707870 22135 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:54.707940 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:54.708137 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:54.710510 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "98a8254006fd4825b0237272a4523d2b"
format_stamp: "Formatted at 2026-08-11 22:14:54 on dist-test-slave-77v9"
server_key: "9343fc8a282cc2e2bec4d5fc12938f9f"
server_key_iv: "b15ffeda4ca3b60c3ecb77921f5e2023"
server_key_version: "encryptionkey@0"
I20260811 22:14:54.711138 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "98a8254006fd4825b0237272a4523d2b"
format_stamp: "Formatted at 2026-08-11 22:14:54 on dist-test-slave-77v9"
server_key: "9343fc8a282cc2e2bec4d5fc12938f9f"
server_key_iv: "b15ffeda4ca3b60c3ecb77921f5e2023"
server_key_version: "encryptionkey@0"
I20260811 22:14:54.715709 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:14:54.718977 22151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:54.720628 22135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:54.720995 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "98a8254006fd4825b0237272a4523d2b"
format_stamp: "Formatted at 2026-08-11 22:14:54 on dist-test-slave-77v9"
server_key: "9343fc8a282cc2e2bec4d5fc12938f9f"
server_key_iv: "b15ffeda4ca3b60c3ecb77921f5e2023"
server_key_version: "encryptionkey@0"
I20260811 22:14:54.721196 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:54.739851 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:54.744108 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:54.744381 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:54.752610 22135 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:43999
I20260811 22:14:54.752640 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:43999 every 8 connection(s)
I20260811 22:14:54.753710 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:54.756956 22210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:54.762413 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22135
I20260811 22:14:54.762543 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:54.762895 18200 external_mini_cluster.cc:1428] Setting key b969d6a00206e8c894eeffd638b9a5b5
I20260811 22:14:54.764037 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b: Bootstrap starting.
I20260811 22:14:54.767030 22210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:54.768188 22210 log.cc:824] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:54.770968 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b: No bootstrap required, opened a new log
Aug 11 22:14:54 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:54.776448 22210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } }
I20260811 22:14:54.776799 22210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:54.776913 22210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a8254006fd4825b0237272a4523d2b, State: Initialized, Role: FOLLOWER
I20260811 22:14:54.777465 22210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } }
I20260811 22:14:54.777617 22210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:54.777716 22210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:54.777838 22210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:54.779414 22210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } }
I20260811 22:14:54.779879 22210 leader_election.cc:302] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98a8254006fd4825b0237272a4523d2b; no voters:
I20260811 22:14:54.780313 22210 leader_election.cc:288] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:54.780441 22215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:54.780781 22215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [term 1 LEADER]: Becoming Leader. State: Replica: 98a8254006fd4825b0237272a4523d2b, State: Running, Role: LEADER
I20260811 22:14:54.781199 22215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } }
I20260811 22:14:54.782130 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:54.764724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.17.198.62:43999)
0811 22:14:54.765298 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:54.765322 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:54.765365 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:14:54.766156 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:54.766177 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:54.766587 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:54.766940 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:54.768084 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:54.769525 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:54.771091 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:54.771639 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:54.771781 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:54.775375 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:54.775382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:54.775402 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:54.775440 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:54.778208 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:54.778888 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:54.778894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:54.778900 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:54.779053 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:54.779509 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:54.779512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:54.779513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:54.780150 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:54.780394 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:54.780796 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:14:54.781219 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":197,"threads_started":1}
I20260811 22:14:54.782968 22216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98a8254006fd4825b0237272a4523d2b. Latest consensus state: current_term: 1 leader_uuid: "98a8254006fd4825b0237272a4523d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } } }
I20260811 22:14:54.783340 22216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:54.783236 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98a8254006fd4825b0237272a4523d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a8254006fd4825b0237272a4523d2b" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43999 } } }
I20260811 22:14:54.783712 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:54.783740 22210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:54.784735 22219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:54.788681 22219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:54.795653 22219 catalog_manager.cc:1347] Generated new cluster ID: 3208ecad5cc94538a42628979250e2d0
I20260811 22:14:54.795768 22219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:54.804028 22219 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:54.805485 22219 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:54.827248 22219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98a8254006fd4825b0237272a4523d2b: Generated new TSK 0
I20260811 22:14:54.828157 22219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:54.873539 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:55.008337 22236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:55.008668 22236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:55.008713 22236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:55.013216 22236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:55.013305 22236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:55.013371 22236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:55.021847 22236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:55.023504 22236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:55.024935 22236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:14:55.036335 22236 server_base.cc:1034] running on GCE node
W20260811 22:14:55.036279 22246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.036540 22242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.036275 22243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:55.037158 22236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:55.037912 22236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:55.039134 22236 hybrid_clock.cc:648] HybridClock initialized: now 1786486495039116 us; error 34 us; skew 500 ppm
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:55.042764 22236 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:55.043911 22236 webserver.cc:460] Webserver started at http://127.17.198.1:33499/ using document root <none> and password file <none>
I20260811 22:14:55.044602 22236 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:55.044715 22236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:55.045046 22236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:55.047660 22236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "71793d1cfca045be9a5d0d4ac89957e5"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "e590819bb2f8ba7adbf0bea15b768aa2"
server_key_iv: "a1973de7b2a50acfb9550f913b81385c"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.048429 22236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "71793d1cfca045be9a5d0d4ac89957e5"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "e590819bb2f8ba7adbf0bea15b768aa2"
server_key_iv: "a1973de7b2a50acfb9550f913b81385c"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.053164 22236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 22:14:55.056151 22253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.057688 22236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:55.057897 22236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "71793d1cfca045be9a5d0d4ac89957e5"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "e590819bb2f8ba7adbf0bea15b768aa2"
server_key_iv: "a1973de7b2a50acfb9550f913b81385c"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.058058 22236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:55.076997 22236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:55.081235 22236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:55.081502 22236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:55.082219 22236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:55.083209 22236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:55.083262 22236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.083339 22236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:55.083382 22236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.093899 22236 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:43215
I20260811 22:14:55.093935 22377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:43215 every 8 connection(s)
I20260811 22:14:55.095050 22236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:55.105150 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22236
I20260811 22:14:55.105337 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:55.105662 18200 external_mini_cluster.cc:1428] Setting key cfbaabb198d29050f1da948b715ca088
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:55.111449 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.097314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:35081 (local address 127.17.198.62:43999)
0811 22:14:55.097497 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.097501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.098386 (+ 885us) server_negotiation.cc:407] Connection header received
0811 22:14:55.099964 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.099968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.100038 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.100158 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.101623 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.102213 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.103480 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.103731 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.103865 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.106769 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.106774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.106777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.106806 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.108657 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.109432 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.109435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.109436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.109527 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.110026 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.110031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.110034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.110330 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.110445 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.111091 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.111261 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:14:55.112684 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.097663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43999 (local address 127.17.198.1:35081)
0811 22:14:55.098179 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:55.098228 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:55.099703 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:55.100368 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:55.100376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:55.100858 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:55.101448 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.101464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.102377 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.102380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.103334 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.103343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.103911 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.103915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.104104 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.104763 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:55.104800 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:55.106589 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:55.108832 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.108844 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.108867 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.109311 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.109630 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.109635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.109640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.109880 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.110478 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:55.110485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:55.110785 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 22:14:55.110978 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 22:14:55.111289 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":139,"threads_started":1}
I20260811 22:14:55.114367 22378 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43999
I20260811 22:14:55.114687 22378 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:55.115443 22378 heartbeater.cc:511] Master 127.17.198.62:43999 requested a full tablet report, sending...
I20260811 22:14:55.117362 22170 ts_manager.cc:194] Registered new tserver with Master: 71793d1cfca045be9a5d0d4ac89957e5 (127.17.198.1:43215)
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:55.118708 22170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:35081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:55.145972 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:55.289602 22383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:55.289942 22383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:55.289990 22383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:55.294504 22383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:55.294601 22383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:55.294665 22383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:55.300578 22383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:55.301855 22383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:55.302953 22383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:55.313748 22389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.313781 22390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.313755 22392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:55.313817 22383 server_base.cc:1034] running on GCE node
I20260811 22:14:55.314556 22383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:55.315330 22383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:55.316514 22383 hybrid_clock.cc:648] HybridClock initialized: now 1786486495316478 us; error 55 us; skew 500 ppm
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:55.319875 22383 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:55.320958 22383 webserver.cc:460] Webserver started at http://127.17.198.2:44519/ using document root <none> and password file <none>
I20260811 22:14:55.321601 22383 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:55.321708 22383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:55.322000 22383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:55.324416 22383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "f27bd91af8744433bb2f99d66648965e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "a3da4b9b3ef3e729b01a117ffc35264c"
server_key_iv: "9660f5aa801be995f24630b7f26cdc3f"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.325139 22383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "f27bd91af8744433bb2f99d66648965e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "a3da4b9b3ef3e729b01a117ffc35264c"
server_key_iv: "9660f5aa801be995f24630b7f26cdc3f"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.329586 22383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:14:55.332489 22399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.334084 22383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:14:55.334271 22383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "f27bd91af8744433bb2f99d66648965e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "a3da4b9b3ef3e729b01a117ffc35264c"
server_key_iv: "9660f5aa801be995f24630b7f26cdc3f"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.334420 22383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:55.348488 22383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:55.352253 22383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:55.352537 22383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:55.353310 22383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:55.354343 22383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:55.354405 22383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.354452 22383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:55.354517 22383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.364574 22383 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:39597
I20260811 22:14:55.364645 22523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:39597 every 8 connection(s)
I20260811 22:14:55.365784 22383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:55.366273 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22383
I20260811 22:14:55.366396 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:55.366668 18200 external_mini_cluster.cc:1428] Setting key 89f061b114d9cd039a303b55d61f0c66
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:55.388549 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.367905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:55183 (local address 127.17.198.62:43999)
0811 22:14:55.368093 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.368098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.369249 (+ 1151us) server_negotiation.cc:407] Connection header received
0811 22:14:55.371240 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.371244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.371307 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.371391 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.372726 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.373635 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.375021 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.375474 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.375625 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.384011 (+ 8386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.384015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.384017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.384049 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.386009 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.386686 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.386692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.386734 (+ 42us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.386847 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.387275 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.387278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.387280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.387506 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.387622 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.388209 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.388373 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:14:55.389554 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.368431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43999 (local address 127.17.198.2:55183)
0811 22:14:55.369047 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:55.369120 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:55.370980 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:55.371595 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:55.371604 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:55.372013 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:55.372551 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.372567 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.373808 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.373811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.374853 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.374866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.375671 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.375674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.375847 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.381428 (+ 5581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:55.381452 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:55.383734 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:55.386181 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.386190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.386211 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.386555 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.386971 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.386975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.386977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.387162 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.387625 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:55.387636 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:55.387944 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 22:14:55.388165 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 22:14:55.388547 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":142,"threads_started":1}
I20260811 22:14:55.391119 22524 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43999
I20260811 22:14:55.391434 22524 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:55.391999 22524 heartbeater.cc:511] Master 127.17.198.62:43999 requested a full tablet report, sending...
I20260811 22:14:55.393311 22170 ts_manager.cc:194] Registered new tserver with Master: f27bd91af8744433bb2f99d66648965e (127.17.198.2:39597)
I20260811 22:14:55.393970 22170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:55183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:55.405611 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:55.537976 22531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:55.538333 22531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:55.538383 22531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:55.542703 22531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:55.542798 22531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:55.542876 22531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:55.550547 22531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:55.552284 22531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:55.553810 22531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:55.568117 22538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.568150 22540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.568158 22537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:55.569360 22531 server_base.cc:1034] running on GCE node
I20260811 22:14:55.570225 22531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:55.571156 22531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:55.572422 22531 hybrid_clock.cc:648] HybridClock initialized: now 1786486495572387 us; error 58 us; skew 500 ppm
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:55.577113 22531 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:55.578408 22531 webserver.cc:460] Webserver started at http://127.17.198.3:41335/ using document root <none> and password file <none>
I20260811 22:14:55.579264 22531 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:55.579391 22531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:55.579674 22531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:55.583091 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "aa745d2f5b00443691eeb6be8f29838e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "191677978ac31bf9e4f863279bff1f4e"
server_key_iv: "b484005194ee0561529a950102ac5f16"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.583875 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "aa745d2f5b00443691eeb6be8f29838e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "191677978ac31bf9e4f863279bff1f4e"
server_key_iv: "b484005194ee0561529a950102ac5f16"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.590270 22531 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:14:55.594367 22547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.596318 22531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:55.596504 22531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "aa745d2f5b00443691eeb6be8f29838e"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-77v9"
server_key: "191677978ac31bf9e4f863279bff1f4e"
server_key_iv: "b484005194ee0561529a950102ac5f16"
server_key_version: "encryptionkey@0"
I20260811 22:14:55.596688 22531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:55.612088 22531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:55.615751 22531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:55.616003 22531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:55.616660 22531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:55.617628 22531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:55.617702 22531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.617779 22531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:55.617821 22531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.626922 22531 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:44329
I20260811 22:14:55.626996 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:44329 every 8 connection(s)
I20260811 22:14:55.627954 22531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:55.635346 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22531
I20260811 22:14:55.635524 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:55.635815 18200 external_mini_cluster.cc:1428] Setting key 333c5dbda0e931d3ced2490db1d53564
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:55.645186 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.630305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:55415 (local address 127.17.198.62:43999)
0811 22:14:55.630453 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.630457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.631454 (+ 997us) server_negotiation.cc:407] Connection header received
0811 22:14:55.633162 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.633167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.633245 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.633381 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.634767 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.635392 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.637214 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.637592 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.637741 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.640682 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.640686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.640688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.640715 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.642209 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.642885 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.642892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.642895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.643026 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.643578 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.643581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.643583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.643845 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.643957 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.644743 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.644964 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:14:55.646376 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.630685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43999 (local address 127.17.198.3:55415)
0811 22:14:55.631258 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:55.631310 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:55.632891 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:55.633582 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:55.633590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:55.634019 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:55.634572 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.634587 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.635557 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.635565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.637017 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:55.637029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.637794 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.637799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:55.638004 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.638759 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:55.638777 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:55.640468 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:55.642380 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.642387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.642412 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.642743 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.643198 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:55.643202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:55.643204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:55.643447 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:55.643982 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:55.643989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:55.644391 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 22:14:55.644642 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:14:55.645096 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":137,"threads_started":1}
I20260811 22:14:55.647742 22669 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43999
I20260811 22:14:55.648066 22669 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:55.648624 22669 heartbeater.cc:511] Master 127.17.198.62:43999 requested a full tablet report, sending...
I20260811 22:14:55.649971 22170 ts_manager.cc:194] Registered new tserver with Master: aa745d2f5b00443691eeb6be8f29838e (127.17.198.3:44329)
I20260811 22:14:55.650619 22170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:55415
I20260811 22:14:55.662696 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:14:55.693495 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.681768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.17.198.1:43215)
0811 22:14:55.682144 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.682151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.682178 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:14:55.682231 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.682236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.682463 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.682624 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.683568 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.684376 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.685621 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.686142 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.686276 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.689437 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.689450 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.689468 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.689504 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.691403 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.691909 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.691915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.691916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.692008 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.692299 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.692304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.692306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.692576 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.692802 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.693137 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.693301 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":130,"threads_started":1}
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:55.706391 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.696790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.17.198.62:43999)
0811 22:14:55.696976 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.696980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.696999 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:14:55.697184 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.697187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.697236 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.697306 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.697960 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.698507 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.699276 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.699487 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.699566 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.702646 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.702649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.702651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.702681 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.704461 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.704948 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.704951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.704952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.705050 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.705396 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.705399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.705400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.705670 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.705781 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.706016 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.706173 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 22:14:55.706971 22170 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57044
I20260811 22:14:55.707721 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22236
I20260811 22:14:55.719102 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22383
I20260811 22:14:55.728039 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22531
I20260811 22:14:55.738498 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22135
2026-08-11T22:14:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2498 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22682](info): set up 2 sockets
Aug 11 22:14:55 dist-test-slave-77v9 krb5kdc[22682](info): commencing operation
krb5kdc: starting...
W20260811 22:14:56.880792 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:14:56 dist-test-slave-77v9 krb5kdc[22682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:14:57.029805 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39587
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:57.148880 22696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:57.149188 22696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:57.149235 22696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:57.153360 22696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:57.153427 22696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:57.153447 22696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:57.153463 22696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:57.153479 22696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:57.158955 22696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:39587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:57.160069 22696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:57.161099 22696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:57.170323 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.170307 22702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.170400 22696 server_base.cc:1034] running on GCE node
W20260811 22:14:57.170344 22704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.171252 22696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:57.172493 22696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:57.173708 22696 hybrid_clock.cc:648] HybridClock initialized: now 1786486497173659 us; error 63 us; skew 500 ppm
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:57.177237 22696 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:14:57.178176 22696 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:57.178289 22696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:57.178596 22696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:57.181079 22696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "86170cefd98e40e99164aaa372259c3d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "80f6f5a1986de5b673df182bcfd60be4"
server_key_iv: "0c3ae285a36a7e75b34e9b3b2d8750e4"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.181754 22696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "86170cefd98e40e99164aaa372259c3d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "80f6f5a1986de5b673df182bcfd60be4"
server_key_iv: "0c3ae285a36a7e75b34e9b3b2d8750e4"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.186106 22696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:57.189595 22712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.191092 22696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:57.191371 22696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "86170cefd98e40e99164aaa372259c3d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "80f6f5a1986de5b673df182bcfd60be4"
server_key_iv: "0c3ae285a36a7e75b34e9b3b2d8750e4"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.191521 22696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:57.212389 22696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:57.215649 22696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:57.215886 22696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:57.223402 22696 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:39587
I20260811 22:14:57.223686 22771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:39587 every 8 connection(s)
I20260811 22:14:57.224179 22696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:57.227088 22772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:57.228592 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22696
I20260811 22:14:57.228739 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:57.229082 18200 external_mini_cluster.cc:1428] Setting key aadcdf8bb247cf9c59f53201e5fc21ce
I20260811 22:14:57.234205 22772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: Bootstrap starting.
I20260811 22:14:57.237190 22772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:57.238227 22772 log.cc:824] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:57.240774 22772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: No bootstrap required, opened a new log
I20260811 22:14:57.244591 22772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } }
I20260811 22:14:57.244791 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.230570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.17.198.62:39587)
0811 22:14:57.231017 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:57.231033 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:57.231069 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:14:57.231709 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:57.231725 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:57.232056 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:57.232345 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:57.233399 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.234568 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.235815 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.236315 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.236447 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.239690 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:57.239696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:57.239711 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:57.239740 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:57.241741 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.242246 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.242255 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.242266 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.242391 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.242736 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.242739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.242741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.243189 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:57.243411 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:57.243705 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:14:57.244029 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":162,"threads_started":1}
I20260811 22:14:57.244871 22772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:57.244997 22772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86170cefd98e40e99164aaa372259c3d, State: Initialized, Role: FOLLOWER
I20260811 22:14:57.245442 22772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } }
I20260811 22:14:57.245568 22772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:57.245656 22772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:57.245757 22772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:57.247049 22772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } }
I20260811 22:14:57.247489 22772 leader_election.cc:302] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86170cefd98e40e99164aaa372259c3d; no voters:
I20260811 22:14:57.247860 22772 leader_election.cc:288] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:57.248106 22778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:57.248425 22778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [term 1 LEADER]: Becoming Leader. State: Replica: 86170cefd98e40e99164aaa372259c3d, State: Running, Role: LEADER
I20260811 22:14:57.248865 22778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } }
I20260811 22:14:57.249271 22772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:57.250636 22779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86170cefd98e40e99164aaa372259c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } } }
I20260811 22:14:57.250804 22779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:57.251313 22789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:57.251258 22780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86170cefd98e40e99164aaa372259c3d. Latest consensus state: current_term: 1 leader_uuid: "86170cefd98e40e99164aaa372259c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86170cefd98e40e99164aaa372259c3d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39587 } } }
I20260811 22:14:57.251385 22780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:57.254525 22789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:57.261183 22789 catalog_manager.cc:1347] Generated new cluster ID: af8bcfca66da4fc7ac1a1a1bcdef693e
I20260811 22:14:57.261297 22789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:57.281028 22789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:57.282115 22789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:57.289014 22789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: Generated new TSK 0
I20260811 22:14:57.289803 22789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:14:57.328657 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:57.454751 22800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:57.455083 22800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:57.455194 22800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:57.459849 22800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:57.459968 22800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:57.460039 22800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:57.466862 22800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:57.468303 22800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:57.469400 22800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:57.480072 22805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.480072 22808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.480558 22800 server_base.cc:1034] running on GCE node
W20260811 22:14:57.480571 22806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.481117 22800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:57.481882 22800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:57.483085 22800 hybrid_clock.cc:648] HybridClock initialized: now 1786486497483066 us; error 31 us; skew 500 ppm
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:57.490389 22800 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:14:57.491240 22800 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:57.491315 22800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:57.491492 22800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:57.493767 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "47385b28e80e4c0985de7c70f56639c5"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "4e9247c80ed398294ea9293435795ba6"
server_key_iv: "3ad45bd74b49c7a7afa953448017d24e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.494323 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "47385b28e80e4c0985de7c70f56639c5"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "4e9247c80ed398294ea9293435795ba6"
server_key_iv: "3ad45bd74b49c7a7afa953448017d24e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.498962 22800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:57.501798 22814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.503321 22800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:57.503494 22800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "47385b28e80e4c0985de7c70f56639c5"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "4e9247c80ed398294ea9293435795ba6"
server_key_iv: "3ad45bd74b49c7a7afa953448017d24e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.503644 22800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:57.532779 22800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:57.536358 22800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:57.536607 22800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:57.537336 22800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:57.538270 22800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:57.538348 22800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.538414 22800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:57.538463 22800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.548259 22800 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:36257
I20260811 22:14:57.548278 22937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:36257 every 8 connection(s)
I20260811 22:14:57.549057 22800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:57.557412 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22800
I20260811 22:14:57.557580 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:57.557926 18200 external_mini_cluster.cc:1428] Setting key 64b86de224f9b2036483031e1f53718c
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:57.564973 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.551085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:36005 (local address 127.17.198.62:39587)
0811 22:14:57.551256 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:57.551261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:57.552695 (+ 1434us) server_negotiation.cc:407] Connection header received
0811 22:14:57.553683 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:57.553686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:57.553737 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:57.553820 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:57.555126 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.555695 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.556965 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.557214 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.557300 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.560301 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:57.560307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:57.560313 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:57.560356 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:57.562274 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.563104 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.563107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.563108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.563192 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.563669 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.563672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.563673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.563887 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:57.564047 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:57.564613 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 22:14:57.564759 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:14:57.565932 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.551601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39587 (local address 127.17.198.1:36005)
0811 22:14:57.552516 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:57.552564 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:57.553469 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:57.553999 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:57.554006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:57.554388 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:57.554953 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.554967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.555826 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.555829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.556755 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.556766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.557393 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.557400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.557674 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.558476 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:57.558497 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:57.560113 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:57.562468 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.562483 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.562510 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.562980 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.563303 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.563308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.563310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.563552 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.564046 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:57.564055 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:57.564345 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 22:14:57.564558 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:14:57.564960 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":148,"threads_started":1}
I20260811 22:14:57.567402 22938 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39587
I20260811 22:14:57.567744 22938 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:57.568408 22938 heartbeater.cc:511] Master 127.17.198.62:39587 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:14:57.570356 22731 ts_manager.cc:194] Registered new tserver with Master: 47385b28e80e4c0985de7c70f56639c5 (127.17.198.1:36257)
I20260811 22:14:57.571594 22731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:36005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:14:57.593780 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:57.715893 22943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:57.716219 22943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:57.716269 22943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:57.720602 22943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:57.720700 22943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:57.720765 22943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:57.727074 22943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:57.728300 22943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:57.729465 22943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:57.739738 22951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.739738 22948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.739782 22943 server_base.cc:1034] running on GCE node
W20260811 22:14:57.739738 22949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.740510 22943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:57.741292 22943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:57.742527 22943 hybrid_clock.cc:648] HybridClock initialized: now 1786486497742480 us; error 53 us; skew 500 ppm
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:57.746053 22943 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:14:57.746979 22943 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:57.747089 22943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:57.747345 22943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:57.749709 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "10fe05caf05549ed8dc63cef330ddf65"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "2b233efe96b20926da33ded63f948679"
server_key_iv: "2a4e85165b1946c82f438b0896228c0e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.750401 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "10fe05caf05549ed8dc63cef330ddf65"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "2b233efe96b20926da33ded63f948679"
server_key_iv: "2a4e85165b1946c82f438b0896228c0e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.754765 22943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:57.757877 22957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.759328 22943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:57.759509 22943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "10fe05caf05549ed8dc63cef330ddf65"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-77v9"
server_key: "2b233efe96b20926da33ded63f948679"
server_key_iv: "2a4e85165b1946c82f438b0896228c0e"
server_key_version: "encryptionkey@0"
I20260811 22:14:57.759658 22943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:57.790787 22943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:57.795197 22943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:57.795462 22943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:57.796226 22943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:57.797345 22943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:57.797423 22943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.797495 22943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:57.797535 22943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:57.807778 22943 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:35649
I20260811 22:14:57.807832 23085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:35649 every 8 connection(s)
I20260811 22:14:57.808501 22943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:57.813181 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 22943
I20260811 22:14:57.813323 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:57.813663 18200 external_mini_cluster.cc:1428] Setting key 010914d4bc98230cf019f4fc15beac53
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:14:57.827478 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.810647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:49191 (local address 127.17.198.62:39587)
0811 22:14:57.810854 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:57.810858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:57.811751 (+ 893us) server_negotiation.cc:407] Connection header received
0811 22:14:57.813355 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:57.813362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:57.813434 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:57.813785 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:57.815252 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.816030 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.817546 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.818072 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.818210 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.821945 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:57.821948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:57.821951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:57.821981 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:57.824002 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.825031 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.825035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.825036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.825157 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.825686 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.825689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.825691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.825979 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:57.826075 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:57.827201 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0811 22:14:57.827340 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:14:57.828361 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.811051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39587 (local address 127.17.198.2:49191)
0811 22:14:57.811572 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:57.811621 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:57.813014 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:57.813994 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:57.814004 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:57.814422 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:57.815087 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.815109 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.816192 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.816195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.817403 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.817414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.818295 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.818301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.818492 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.819289 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:57.819317 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:57.821602 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:57.824176 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.824188 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.824218 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.824724 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.825278 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.825284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.825289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.825558 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.826111 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:57.826119 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:57.826636 (+ 517us) client_negotiation.cc:769] Sending connection context
0811 22:14:57.826888 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 22:14:57.827228 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":122,"threads_started":1}
I20260811 22:14:57.830050 23087 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:14:57.830368 23087 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:57.831029 23087 heartbeater.cc:511] Master 127.17.198.62:39587 requested a full tablet report, sending...
I20260811 22:14:57.832368 22731 ts_manager.cc:194] Registered new tserver with Master: 10fe05caf05549ed8dc63cef330ddf65 (127.17.198.2:35649)
I20260811 22:14:57.833127 22731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:49191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:14:57.849095 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:57.974943 23092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:57.975274 23092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:57.975332 23092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:57.979689 23092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:57.979779 23092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:57.979844 23092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:57.985612 23092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:57.986742 23092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:57.987862 23092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:57.998306 23097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.998323 23100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.998330 23098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.998482 23092 server_base.cc:1034] running on GCE node
I20260811 22:14:57.999127 23092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:57.999791 23092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:58.000963 23092 hybrid_clock.cc:648] HybridClock initialized: now 1786486498000904 us; error 69 us; skew 500 ppm
Aug 11 22:14:57 dist-test-slave-77v9 krb5kdc[22682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:58.004432 23092 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:14:58.005505 23092 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:58.005616 23092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:58.005892 23092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:58.008538 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "a70149fe83884299a2d31ac8cb66dbcc"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "24d813af919bcce0aa525e824d12d97a"
server_key_iv: "7a26903f174f43f92d28f742347b4bf0"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.009280 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "a70149fe83884299a2d31ac8cb66dbcc"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "24d813af919bcce0aa525e824d12d97a"
server_key_iv: "7a26903f174f43f92d28f742347b4bf0"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.013779 23092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:58.016726 23106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.018430 23092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:58.018605 23092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "a70149fe83884299a2d31ac8cb66dbcc"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "24d813af919bcce0aa525e824d12d97a"
server_key_iv: "7a26903f174f43f92d28f742347b4bf0"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.018756 23092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.032773 23092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.037114 23092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.037374 23092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.038089 23092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:58.039076 23092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:58.039131 23092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.039177 23092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:58.039194 23092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.049477 23092 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:39675
I20260811 22:14:58.049563 23233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:39675 every 8 connection(s)
I20260811 22:14:58.050355 23092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:58.059006 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23092
I20260811 22:14:58.059182 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:58.059626 18200 external_mini_cluster.cc:1428] Setting key 0ef23985bbb1e6ca807874a86738f350
Aug 11 22:14:58 dist-test-slave-77v9 krb5kdc[22682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:14:58.067217 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.052459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:55275 (local address 127.17.198.62:39587)
0811 22:14:58.052628 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.052633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.054011 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 22:14:58.054928 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.054931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.054985 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.055076 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.056363 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.057030 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.058432 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.058660 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.058760 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.062483 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.062487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.062490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.062527 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.064546 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.065204 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.065207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.065208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.065309 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.065683 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.065686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.065687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.065977 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.066068 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.066923 (+ 855us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.067077 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:14:58.068087 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.052945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39587 (local address 127.17.198.3:55275)
0811 22:14:58.053839 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.053888 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.054688 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.055246 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.055253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.055672 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:58.056176 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.056190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.057182 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.057186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.058262 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.058273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.058971 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.058979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.059233 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.060035 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:58.060063 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:58.062229 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:58.064717 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.064723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.064738 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.065092 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.065403 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.065405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.065406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.065579 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.066113 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:58.066120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:58.066495 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.066774 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.067126 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":118,"threads_started":1}
I20260811 22:14:58.069653 23234 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39587
I20260811 22:14:58.069926 23234 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:58.070436 23234 heartbeater.cc:511] Master 127.17.198.62:39587 requested a full tablet report, sending...
I20260811 22:14:58.071606 22731 ts_manager.cc:194] Registered new tserver with Master: a70149fe83884299a2d31ac8cb66dbcc (127.17.198.3:39675)
I20260811 22:14:58.072271 22731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:55275
I20260811 22:14:58.074947 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:58.086367 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.077353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.17.198.62:39587)
0811 22:14:58.077528 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.077535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.077753 (+ 218us) server_negotiation.cc:407] Connection header received
0811 22:14:58.077887 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.077890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.077946 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.078037 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.078848 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.079373 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.080247 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.080481 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.080607 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.082420 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.082423 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.082425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.082456 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.084124 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.084739 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.084750 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.084752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.084916 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.085248 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.085253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.085255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.085588 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.085760 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.086039 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.086180 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:14:58.089977 22731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:58.093531 22731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:58.111542 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.105529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:39675 (local address 127.0.0.1:46392)
0811 22:14:58.105853 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.105867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.105980 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.106413 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.106416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.106439 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:58.106572 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.106579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.108001 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.108004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.109551 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.109562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.110644 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.110648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.110789 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.110839 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.110976 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.111336 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":147,"thread_start_us":102,"threads_started":1}
I20260811 22:14:58.111888 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.104300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36257 (local address 127.0.0.1:43846)
0811 22:14:58.105324 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.105376 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.105488 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.106508 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.106512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.106531 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:58.106644 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.106653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.108272 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.108278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.110032 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.110044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.111391 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.111396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.111493 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.111521 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.111616 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.111695 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":213,"thread_start_us":136,"threads_started":1}
I20260811 22:14:58.112217 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.104638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.17.198.2:35649)
0811 22:14:58.105172 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.105185 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.105662 (+ 477us) server_negotiation.cc:407] Connection header received
0811 22:14:58.105760 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.105767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.106013 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.106187 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:58.106557 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.107939 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.110202 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.110975 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.111077 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.111212 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.111637 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.112020 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":171,"threads_started":1}
I20260811 22:14:58.112356 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.104644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.17.198.1:36257)
0811 22:14:58.105821 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.105828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.105849 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:14:58.105915 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.105919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.106148 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.106302 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:58.106789 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.108138 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.110178 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.111064 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.111179 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.111311 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.112061 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.112186 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":113,"threads_started":1}
I20260811 22:14:58.112641 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.104087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:35649 (local address 127.0.0.1:50518)
0811 22:14:58.105246 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.105314 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.105487 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.106199 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.106205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.106228 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:58.106390 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.106399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.108129 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.108133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.110036 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.110044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.111287 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.111291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.111378 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.111401 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.111477 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.111546 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":268,"thread_start_us":108,"threads_started":1}
I20260811 22:14:58.112673 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.105117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.17.198.3:39675)
0811 22:14:58.105726 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.105741 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.105986 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:14:58.106091 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.106095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.106269 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.106409 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:58.106698 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.107823 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.109703 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.110494 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.110616 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.110778 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.111038 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.111173 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":190,"threads_started":1}
I20260811 22:14:58.115561 22865 tablet_service.cc:1467] Processing CreateTablet for tablet a181366be63f4605a68660f067a10435 (DEFAULT_TABLE table=test-table [id=14f34d9c279e44c99c0f1ec5e1c22da7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:58.115594 23158 tablet_service.cc:1467] Processing CreateTablet for tablet a181366be63f4605a68660f067a10435 (DEFAULT_TABLE table=test-table [id=14f34d9c279e44c99c0f1ec5e1c22da7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:58.115600 23013 tablet_service.cc:1467] Processing CreateTablet for tablet a181366be63f4605a68660f067a10435 (DEFAULT_TABLE table=test-table [id=14f34d9c279e44c99c0f1ec5e1c22da7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:58.116744 22865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181366be63f4605a68660f067a10435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:58.116772 23013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181366be63f4605a68660f067a10435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:58.116778 23158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a181366be63f4605a68660f067a10435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:58.124054 23251 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: Bootstrap starting.
I20260811 22:14:58.124747 23252 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Bootstrap starting.
I20260811 22:14:58.125748 23253 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Bootstrap starting.
I20260811 22:14:58.127225 23252 tablet_bootstrap.cc:654] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:58.127552 23251 tablet_bootstrap.cc:654] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:58.127965 23253 tablet_bootstrap.cc:654] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:58.128657 23251 log.cc:824] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:58.128687 23252 log.cc:824] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:58.129106 23253 log.cc:824] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:58.131034 23252 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: No bootstrap required, opened a new log
I20260811 22:14:58.131199 23253 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: No bootstrap required, opened a new log
I20260811 22:14:58.131199 23251 tablet_bootstrap.cc:492] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: No bootstrap required, opened a new log
I20260811 22:14:58.131337 23252 ts_tablet_manager.cc:1397] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:14:58.131421 23253 ts_tablet_manager.cc:1397] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:14:58.131421 23251 ts_tablet_manager.cc:1397] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:14:58.135277 23252 raft_consensus.cc:348] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.135646 23252 raft_consensus.cc:374] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:58.135713 23252 raft_consensus.cc:729] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fe05caf05549ed8dc63cef330ddf65, State: Initialized, Role: FOLLOWER
I20260811 22:14:58.136241 23252 consensus_queue.cc:260] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.136411 23253 raft_consensus.cc:348] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.136502 23251 raft_consensus.cc:348] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.136739 23251 raft_consensus.cc:374] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:58.136737 23253 raft_consensus.cc:374] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:58.136831 23253 raft_consensus.cc:729] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a70149fe83884299a2d31ac8cb66dbcc, State: Initialized, Role: FOLLOWER
I20260811 22:14:58.136834 23251 raft_consensus.cc:729] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47385b28e80e4c0985de7c70f56639c5, State: Initialized, Role: FOLLOWER
I20260811 22:14:58.137246 23087 heartbeater.cc:503] Master 127.17.198.62:39587 was elected leader, sending a full tablet report...
I20260811 22:14:58.137377 23253 consensus_queue.cc:260] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.137377 23251 consensus_queue.cc:260] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.137548 23252 ts_tablet_manager.cc:1428] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 22:14:58.138330 23234 heartbeater.cc:503] Master 127.17.198.62:39587 was elected leader, sending a full tablet report...
I20260811 22:14:58.138465 23253 ts_tablet_manager.cc:1428] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:14:58.138703 22938 heartbeater.cc:503] Master 127.17.198.62:39587 was elected leader, sending a full tablet report...
I20260811 22:14:58.140115 23251 ts_tablet_manager.cc:1428] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 22:14:58.162959 23258 raft_consensus.cc:482] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:58.163174 23258 raft_consensus.cc:504] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.164638 23258 leader_election.cc:288] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10fe05caf05549ed8dc63cef330ddf65 (127.17.198.2:35649), 47385b28e80e4c0985de7c70f56639c5 (127.17.198.1:36257)
I20260811 22:14:58.169349 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.164992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36257 (local address 127.17.198.3:51697)
0811 22:14:58.165224 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.165243 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.165396 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.165722 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.165726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.165746 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:58.165902 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.165911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.166885 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.166889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.167976 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.167987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.168795 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.168799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.168949 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.168979 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.169060 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.169146 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3}
I20260811 22:14:58.169425 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.165076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:51697 (local address 127.17.198.1:36257)
0811 22:14:58.165209 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.165214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.165331 (+ 117us) server_negotiation.cc:407] Connection header received
0811 22:14:58.165524 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.165531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.165596 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.165688 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:58.166055 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.166733 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.168120 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.168655 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.168754 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.168803 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.169092 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.169289 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:14:58.169867 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.165076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:36441 (local address 127.17.198.2:35649)
0811 22:14:58.165209 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.165214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.165612 (+ 398us) server_negotiation.cc:407] Connection header received
0811 22:14:58.165755 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.165758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.165808 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.165909 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:58.166325 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.167003 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.168416 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.169104 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.169242 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.169289 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.169661 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.169746 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:14:58.170231 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.165057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:35649 (local address 127.17.198.3:36441)
0811 22:14:58.165494 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.165508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.165626 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.165950 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.165953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.166008 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:58.166156 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.166164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.167145 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.167148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.168272 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.168282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.169378 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.169382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.169489 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.169519 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.170052 (+ 533us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.170134 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":173,"threads_started":1}
I20260811 22:14:58.170332 22888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181366be63f4605a68660f067a10435" candidate_uuid: "a70149fe83884299a2d31ac8cb66dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47385b28e80e4c0985de7c70f56639c5" is_pre_election: true
I20260811 22:14:58.170704 22888 raft_consensus.cc:2454] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a70149fe83884299a2d31ac8cb66dbcc in term 0.
I20260811 22:14:58.170847 23037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181366be63f4605a68660f067a10435" candidate_uuid: "a70149fe83884299a2d31ac8cb66dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fe05caf05549ed8dc63cef330ddf65" is_pre_election: true
I20260811 22:14:58.171114 23037 raft_consensus.cc:2454] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a70149fe83884299a2d31ac8cb66dbcc in term 0.
I20260811 22:14:58.171306 23113 leader_election.cc:302] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47385b28e80e4c0985de7c70f56639c5, a70149fe83884299a2d31ac8cb66dbcc; no voters:
I20260811 22:14:58.171625 23258 raft_consensus.cc:2781] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:58.171705 23258 raft_consensus.cc:482] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:58.171737 23258 raft_consensus.cc:3037] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:58.172953 23258 raft_consensus.cc:504] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.173398 23258 leader_election.cc:288] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [CANDIDATE]: Term 1 election: Requested vote from peers 10fe05caf05549ed8dc63cef330ddf65 (127.17.198.2:35649), 47385b28e80e4c0985de7c70f56639c5 (127.17.198.1:36257)
I20260811 22:14:58.173774 23037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181366be63f4605a68660f067a10435" candidate_uuid: "a70149fe83884299a2d31ac8cb66dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fe05caf05549ed8dc63cef330ddf65"
I20260811 22:14:58.173947 23037 raft_consensus.cc:3037] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:58.173951 22888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a181366be63f4605a68660f067a10435" candidate_uuid: "a70149fe83884299a2d31ac8cb66dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47385b28e80e4c0985de7c70f56639c5"
I20260811 22:14:58.174152 22888 raft_consensus.cc:3037] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:58.175247 23037 raft_consensus.cc:2454] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a70149fe83884299a2d31ac8cb66dbcc in term 1.
I20260811 22:14:58.175635 22888 raft_consensus.cc:2454] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a70149fe83884299a2d31ac8cb66dbcc in term 1.
I20260811 22:14:58.175650 23108 leader_election.cc:302] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fe05caf05549ed8dc63cef330ddf65, a70149fe83884299a2d31ac8cb66dbcc; no voters:
I20260811 22:14:58.176024 23258 raft_consensus.cc:2781] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:58.176343 23258 raft_consensus.cc:686] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 1 LEADER]: Becoming Leader. State: Replica: a70149fe83884299a2d31ac8cb66dbcc, State: Running, Role: LEADER
I20260811 22:14:58.176797 23258 consensus_queue.cc:237] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } }
I20260811 22:14:58.180567 22729 catalog_manager.cc:5495] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc reported cstate change: term changed from 0 to 1, leader changed from <none> to a70149fe83884299a2d31ac8cb66dbcc (127.17.198.3). New cstate: current_term: 1 leader_uuid: "a70149fe83884299a2d31ac8cb66dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a70149fe83884299a2d31ac8cb66dbcc" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 39675 } health_report { overall_health: HEALTHY } } }
I20260811 22:14:58.201092 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.196660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.17.198.3:39675)
0811 22:14:58.196862 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.196867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.196885 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:14:58.197115 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.197118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.197181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.197285 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:58.197623 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.198299 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.199344 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.199609 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.199699 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.200046 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:58.200534 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:58.200635 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.200760 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.200885 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:14:58.207864 23037 raft_consensus.cc:1261] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 1 FOLLOWER]: Refusing update from remote peer a70149fe83884299a2d31ac8cb66dbcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:58.207911 22888 raft_consensus.cc:1261] T a181366be63f4605a68660f067a10435 P 47385b28e80e4c0985de7c70f56639c5 [term 1 FOLLOWER]: Refusing update from remote peer a70149fe83884299a2d31ac8cb66dbcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:58.208670 23258 consensus_queue.cc:1035] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "47385b28e80e4c0985de7c70f56639c5" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:58.208945 23261 consensus_queue.cc:1035] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "10fe05caf05549ed8dc63cef330ddf65" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:58.216910 23266 mvcc.cc:205] Tried to move back new op lower bound from 7317448696650715136 to 7317448696535277568. Current Snapshot: MvccSnapshot[applied={T|T < 7317448696650715136}]
I20260811 22:14:58.218477 23267 mvcc.cc:205] Tried to move back new op lower bound from 7317448696650715136 to 7317448696535277568. Current Snapshot: MvccSnapshot[applied={T|T < 7317448696650715136}]
I20260811 22:14:58.226313 22729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:58.226540 22729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:58.229282 22729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86170cefd98e40e99164aaa372259c3d: Sending DeleteTablet for 3 replicas of tablet a181366be63f4605a68660f067a10435
I20260811 22:14:58.230345 23013 tablet_service.cc:1514] Processing DeleteTablet for tablet a181366be63f4605a68660f067a10435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:58 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50518
I20260811 22:14:58.230458 22865 tablet_service.cc:1514] Processing DeleteTablet for tablet a181366be63f4605a68660f067a10435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:58 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:43846
I20260811 22:14:58.230993 23277 tablet_replica.cc:330] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: stopping tablet replica
I20260811 22:14:58.231220 23158 tablet_service.cc:1514] Processing DeleteTablet for tablet a181366be63f4605a68660f067a10435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:58 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46392
I20260811 22:14:58.231555 23277 raft_consensus.cc:2229] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:58.232030 23277 raft_consensus.cc:2258] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:58.232427 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22800
I20260811 22:14:58.241016 23279 tablet_replica.cc:330] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: stopping tablet replica
I20260811 22:14:58.241433 23279 raft_consensus.cc:2229] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:58.241936 23279 raft_consensus.cc:2258] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:58.241976 23277 ts_tablet_manager.cc:1905] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:58.243402 23279 ts_tablet_manager.cc:1905] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:58.245419 23277 ts_tablet_manager.cc:1918] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:58.245641 23277 log.cc:1196] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/a181366be63f4605a68660f067a10435
I20260811 22:14:58.246141 23277 ts_tablet_manager.cc:1939] T a181366be63f4605a68660f067a10435 P 10fe05caf05549ed8dc63cef330ddf65: Deleting consensus metadata
I20260811 22:14:58.246668 23279 ts_tablet_manager.cc:1918] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:58.246785 23279 log.cc:1196] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/a181366be63f4605a68660f067a10435
I20260811 22:14:58.247180 23279 ts_tablet_manager.cc:1939] T a181366be63f4605a68660f067a10435 P a70149fe83884299a2d31ac8cb66dbcc: Deleting consensus metadata
W20260811 22:14:58.247689 22716 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:36257 (error 108)
I20260811 22:14:58.248183 22713 catalog_manager.cc:4841] TS 10fe05caf05549ed8dc63cef330ddf65 (127.17.198.2:35649): tablet a181366be63f4605a68660f067a10435 (table test-table [id=14f34d9c279e44c99c0f1ec5e1c22da7]) successfully deleted
I20260811 22:14:58.248422 22713 catalog_manager.cc:4841] TS a70149fe83884299a2d31ac8cb66dbcc (127.17.198.3:39675): tablet a181366be63f4605a68660f067a10435 (table test-table [id=14f34d9c279e44c99c0f1ec5e1c22da7]) successfully deleted
I20260811 22:14:58.248692 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22943
I20260811 22:14:58.262135 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.261745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36257 (local address 127.0.0.1:43862)
0811 22:14:58.261927 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.262026 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 22:14:58.262502 22716 catalog_manager.cc:4584] TS 47385b28e80e4c0985de7c70f56639c5 (127.17.198.1:36257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a181366be63f4605a68660f067a10435: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36257: connect: Connection refused (error 111)
I20260811 22:14:58.262686 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23092
I20260811 22:14:58.271776 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 22696
2026-08-11T22:14:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2534 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:58Z Disabled control of system clock
I20260811 22:14:58.309098 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43491
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:33329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:43491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:14:58.431313 23284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:58.431638 23284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:58.431679 23284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:58.436096 23284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:58.436177 23284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:58.436193 23284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:58.436211 23284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:58.436228 23284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:58.441429 23284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:43491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:58.442716 23284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:58.444224 23284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:58.453547 23290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.453563 23289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.453581 23292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.454116 23284 server_base.cc:1034] running on GCE node
I20260811 22:14:58.454716 23284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:58.455951 23284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:58.457122 23284 hybrid_clock.cc:648] HybridClock initialized: now 1786486498457098 us; error 34 us; skew 500 ppm
I20260811 22:14:58.459304 23284 webserver.cc:460] Webserver started at http://127.17.198.62:44469/ using document root <none> and password file <none>
I20260811 22:14:58.459940 23284 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:58.460045 23284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:58.460294 23284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:58.462620 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "7f7c26ee5fb14eb59f4abb020d876021"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "9ce2b0437403a85a97e1aec4023b58b4"
server_key_iv: "e235ecf332a639f31e8e4a9050538de9"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.463239 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "7f7c26ee5fb14eb59f4abb020d876021"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "9ce2b0437403a85a97e1aec4023b58b4"
server_key_iv: "e235ecf332a639f31e8e4a9050538de9"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.467417 23284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:58.470461 23298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.471859 23284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:58.472038 23284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "7f7c26ee5fb14eb59f4abb020d876021"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "9ce2b0437403a85a97e1aec4023b58b4"
server_key_iv: "e235ecf332a639f31e8e4a9050538de9"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.472186 23284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.477479 23284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.478184 23284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.478394 23284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.486191 23284 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:43491
I20260811 22:14:58.486230 23356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:43491 every 8 connection(s)
I20260811 22:14:58.487272 23284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:58.491403 23357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:58.496591 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23284
I20260811 22:14:58.496709 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:58.497049 18200 external_mini_cluster.cc:1428] Setting key b6c89a695e298270bdcb84ee2811729e
I20260811 22:14:58.498974 23357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: Bootstrap starting.
I20260811 22:14:58.501667 23357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:58.502547 23357 log.cc:824] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:58.504077 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.498850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.17.198.62:43491)
0811 22:14:58.499639 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.499662 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.499713 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:14:58.500531 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.500551 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.501038 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.501441 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.502418 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.502423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.502437 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:58.502450 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.502513 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:58.502531 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.502680 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.502799 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.503126 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":162,"threads_started":1}
I20260811 22:14:58.505405 23357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: No bootstrap required, opened a new log
I20260811 22:14:58.510031 23357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } }
I20260811 22:14:58.510356 23357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:58.510453 23357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7c26ee5fb14eb59f4abb020d876021, State: Initialized, Role: FOLLOWER
I20260811 22:14:58.510989 23357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } }
I20260811 22:14:58.511163 23357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:58.511242 23357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:58.511363 23357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:58.512528 23357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } }
I20260811 22:14:58.513022 23357 leader_election.cc:302] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f7c26ee5fb14eb59f4abb020d876021; no voters:
I20260811 22:14:58.513423 23357 leader_election.cc:288] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:58.513561 23362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:58.513828 23362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [term 1 LEADER]: Becoming Leader. State: Replica: 7f7c26ee5fb14eb59f4abb020d876021, State: Running, Role: LEADER
I20260811 22:14:58.514205 23362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } }
I20260811 22:14:58.514878 23357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:58.515969 23363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f7c26ee5fb14eb59f4abb020d876021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } } }
I20260811 22:14:58.516130 23363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:58.516393 23364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f7c26ee5fb14eb59f4abb020d876021. Latest consensus state: current_term: 1 leader_uuid: "7f7c26ee5fb14eb59f4abb020d876021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c26ee5fb14eb59f4abb020d876021" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43491 } } }
I20260811 22:14:58.516486 23364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:58.516505 23372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:58.519364 23372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:58.525990 23372 catalog_manager.cc:1347] Generated new cluster ID: 5a6dc2cb9a0442f68ed78f89090e012d
I20260811 22:14:58.526084 23372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:58.537588 23372 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:58.538008 23372 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:58.549422 23372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: Generated new TSK 0
I20260811 22:14:58.550369 23372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:58.560443 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--builtin_ntp_servers=127.17.198.20:33329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:14:58.687005 23382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:58.687342 23382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:58.687388 23382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:58.692305 23382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:58.692399 23382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:58.692463 23382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:14:58.697742 23382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:58.699087 23382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:58.700573 23382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:58.710842 23387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.710888 23382 server_base.cc:1034] running on GCE node
W20260811 22:14:58.710846 23388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.710920 23392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.711475 23382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:58.712167 23382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:58.713377 23382 hybrid_clock.cc:648] HybridClock initialized: now 1786486498713354 us; error 40 us; skew 500 ppm
I20260811 22:14:58.715639 23382 webserver.cc:460] Webserver started at http://127.17.198.1:33167/ using document root <none> and password file <none>
I20260811 22:14:58.716315 23382 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:58.716411 23382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:58.716658 23382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:58.718915 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "a2b33cc74a14465ab9a8da7139b20047"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "e51a40f5e0ce74214cb0d8dd51000dfd"
server_key_iv: "1637005b19ec0c87fba12ca3597c5bcd"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.719521 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "a2b33cc74a14465ab9a8da7139b20047"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "e51a40f5e0ce74214cb0d8dd51000dfd"
server_key_iv: "1637005b19ec0c87fba12ca3597c5bcd"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.724223 23382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:14:58.727260 23398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.728972 23382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:58.729111 23382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "a2b33cc74a14465ab9a8da7139b20047"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "e51a40f5e0ce74214cb0d8dd51000dfd"
server_key_iv: "1637005b19ec0c87fba12ca3597c5bcd"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.729247 23382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.734338 23382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.735051 23382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.735255 23382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.735877 23382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:58.736928 23382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:58.736999 23382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.737071 23382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:58.737118 23382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.746603 23382 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:43545
I20260811 22:14:58.746634 23525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:43545 every 8 connection(s)
I20260811 22:14:58.747658 23382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:58.748768 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23382
I20260811 22:14:58.748971 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:58.749293 18200 external_mini_cluster.cc:1428] Setting key cf306adfcae45e0b669af2f77b2a27d7
I20260811 22:14:58.751646 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--builtin_ntp_servers=127.17.198.20:33329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:14:58.755815 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.749994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:46667 (local address 127.17.198.62:43491)
0811 22:14:58.750157 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.750162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.751622 (+ 1460us) server_negotiation.cc:407] Connection header received
0811 22:14:58.753846 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.753849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.753899 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.753994 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.755276 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.755279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.755282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:58.755297 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.755327 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:58.755336 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.755439 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.755620 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.755671 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:14:58.756875 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.750488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43491 (local address 127.17.198.1:46667)
0811 22:14:58.751388 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.751489 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.753562 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.754192 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.754199 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.754294 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:58.754996 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:58.754998 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:58.755040 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:58.755112 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:58.755461 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:58.755471 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:58.755483 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.755670 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.755939 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":165,"threads_started":1}
I20260811 22:14:58.758263 23526 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43491
I20260811 22:14:58.758543 23526 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:58.759001 23526 heartbeater.cc:511] Master 127.17.198.62:43491 requested a full tablet report, sending...
I20260811 22:14:58.760988 23315 ts_manager.cc:194] Registered new tserver with Master: a2b33cc74a14465ab9a8da7139b20047 (127.17.198.1:43545)
W20260811 22:14:58.876573 23529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:58.877033 23529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:58.877086 23529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:58.881390 23529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:58.881464 23529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:58.881527 23529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:14:58.886451 23529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:58.887614 23529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:58.889137 23529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:58.899405 23537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.899506 23529 server_base.cc:1034] running on GCE node
W20260811 22:14:58.899358 23536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.899678 23539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.900252 23529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:58.901067 23529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:58.902297 23529 hybrid_clock.cc:648] HybridClock initialized: now 1786486498902275 us; error 36 us; skew 500 ppm
I20260811 22:14:58.904541 23529 webserver.cc:460] Webserver started at http://127.17.198.2:44377/ using document root <none> and password file <none>
I20260811 22:14:58.905254 23529 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:58.905377 23529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:58.905632 23529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:58.908039 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "86b8c0a294d4441da351206a9e122638"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "ca6eb2ff90a8c324599176681d7308a2"
server_key_iv: "ba4ebcdea1c7df1ea58c0f4ff7ecfb59"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.908702 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "86b8c0a294d4441da351206a9e122638"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "ca6eb2ff90a8c324599176681d7308a2"
server_key_iv: "ba4ebcdea1c7df1ea58c0f4ff7ecfb59"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.913053 23529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:58.916208 23546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.917727 23529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:58.917860 23529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "86b8c0a294d4441da351206a9e122638"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-77v9"
server_key: "ca6eb2ff90a8c324599176681d7308a2"
server_key_iv: "ba4ebcdea1c7df1ea58c0f4ff7ecfb59"
server_key_version: "encryptionkey@0"
I20260811 22:14:58.918042 23529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.923321 23529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.924108 23529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.924357 23529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.925058 23529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:58.926106 23529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:58.926156 23529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.926198 23529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:58.926254 23529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.936110 23529 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:34741
I20260811 22:14:58.936146 23675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:34741 every 8 connection(s)
I20260811 22:14:58.937271 23529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:58.940862 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23529
I20260811 22:14:58.941025 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:58.941434 18200 external_mini_cluster.cc:1428] Setting key e04498d5ba82e90e73bb5c4237592288
I20260811 22:14:58.944048 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--builtin_ntp_servers=127.17.198.20:33329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:14:58.945533 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.939478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:34719 (local address 127.17.198.62:43491)
0811 22:14:58.939682 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.939686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.941129 (+ 1443us) server_negotiation.cc:407] Connection header received
0811 22:14:58.943070 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.943076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.943125 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.943252 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.944731 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.944739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.944745 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:58.944763 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.944794 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:58.944845 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.945111 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.945268 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.945330 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 22:14:58.947029 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.939893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43491 (local address 127.17.198.2:34719)
0811 22:14:58.940909 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.940985 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.942777 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.943504 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.943512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.943614 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:58.944435 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:58.944437 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:58.944484 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:58.944563 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:58.945099 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:58.945108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:58.945114 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.945311 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.945560 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":134,"threads_started":1}
I20260811 22:14:58.948553 23676 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43491
I20260811 22:14:58.948936 23676 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:58.949457 23676 heartbeater.cc:511] Master 127.17.198.62:43491 requested a full tablet report, sending...
I20260811 22:14:58.950773 23315 ts_manager.cc:194] Registered new tserver with Master: 86b8c0a294d4441da351206a9e122638 (127.17.198.2:34741)
W20260811 22:14:59.066862 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:59.067234 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:59.067323 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:59.071712 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:59.071846 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:59.071949 23679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:14:59.077289 23679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:59.078470 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:59.079843 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:59.090325 23684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:59.090385 23679 server_base.cc:1034] running on GCE node
W20260811 22:14:59.090317 23687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:59.090317 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:59.091005 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:59.091747 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:59.092935 23679 hybrid_clock.cc:648] HybridClock initialized: now 1786486499092892 us; error 56 us; skew 500 ppm
I20260811 22:14:59.095139 23679 webserver.cc:460] Webserver started at http://127.17.198.3:33201/ using document root <none> and password file <none>
I20260811 22:14:59.095785 23679 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:59.095855 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:59.096033 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:59.098322 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "8234674f906b40719234b4e32de97e91"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "297f99d2f94e3f368eb583b64f7003e7"
server_key_iv: "b465fb02bc0d242e7a1a93ad898eb394"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.098881 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "8234674f906b40719234b4e32de97e91"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "297f99d2f94e3f368eb583b64f7003e7"
server_key_iv: "b465fb02bc0d242e7a1a93ad898eb394"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.103017 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:59.105875 23693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:59.107369 23679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:14:59.107535 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "8234674f906b40719234b4e32de97e91"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "297f99d2f94e3f368eb583b64f7003e7"
server_key_iv: "b465fb02bc0d242e7a1a93ad898eb394"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.107671 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:59.112866 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:59.113608 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:59.113832 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:59.114516 23679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:59.115644 23679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:59.115695 23679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:59.115760 23679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:59.115794 23679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:59.125736 23679 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:46227
I20260811 22:14:59.125758 23818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:46227 every 8 connection(s)
I20260811 22:14:59.126746 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:59.133255 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23679
I20260811 22:14:59.133409 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:59.133734 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.128762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:35485 (local address 127.17.198.62:43491)
0811 22:14:59.128979 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.128983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.130133 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 22:14:59.131588 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.131591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.131645 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.131761 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.133126 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.133129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.133132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.133145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.133188 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.133201 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.133327 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.133572 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.133619 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 22:14:59.133750 18200 external_mini_cluster.cc:1428] Setting key 0355b3f8d364151ca49fa99c655a29cd
I20260811 22:14:59.134999 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.129235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:43491 (local address 127.17.198.3:35485)
0811 22:14:59.129955 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.130000 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.131346 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.131941 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.131948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.132042 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.132745 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.132747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.132791 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.132944 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.133323 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.133343 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.133351 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.133599 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.133886 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":133,"threads_started":1}
I20260811 22:14:59.136258 23821 heartbeater.cc:347] Connected to a master server at 127.17.198.62:43491
I20260811 22:14:59.136540 23821 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:59.137081 23821 heartbeater.cc:511] Master 127.17.198.62:43491 requested a full tablet report, sending...
I20260811 22:14:59.138355 23315 ts_manager.cc:194] Registered new tserver with Master: 8234674f906b40719234b4e32de97e91 (127.17.198.3:46227)
I20260811 22:14:59.148869 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:59.152819 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.151093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.17.198.62:43491)
0811 22:14:59.151258 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.151261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.151443 (+ 182us) server_negotiation.cc:407] Connection header received
0811 22:14:59.151518 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.151521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.151561 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.151638 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.152417 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.152421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.152423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.152435 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.152454 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.152464 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.152544 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.152641 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.152680 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:14:59.156425 23315 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:59.159811 23315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:59.172477 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.169301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:34741 (local address 127.0.0.1:45328)
0811 22:14:59.170038 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.170052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.170117 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.171095 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.171098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.171117 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.171691 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.171692 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.171704 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.171728 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.172118 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.172122 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.172125 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.172209 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.172282 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":139,"threads_started":1}
I20260811 22:14:59.172482 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.169252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:46227 (local address 127.0.0.1:44924)
0811 22:14:59.169690 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.169739 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.169854 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.170748 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.170755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.170831 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.171625 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.171627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.171644 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.171697 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.172080 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.172093 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.172099 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.172204 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.172276 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260811 22:14:59.173152 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.170063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43545 (local address 127.0.0.1:44888)
0811 22:14:59.171431 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.171446 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.171558 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.172002 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.172008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.172027 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.172474 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.172476 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.172487 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.172517 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.172915 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.172918 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.172921 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.172996 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.173029 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"thread_start_us":87,"threads_started":1}
I20260811 22:14:59.173164 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.169413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.17.198.3:46227)
0811 22:14:59.170279 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.170294 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.170321 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:14:59.170390 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.170397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.170605 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.170748 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.171840 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.171852 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.171863 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.171874 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.171912 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.171972 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.172888 (+ 916us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.172947 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.173020 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":159,"threads_started":1}
I20260811 22:14:59.173599 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.170423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.17.198.1:43545)
0811 22:14:59.171004 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.171016 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.171553 (+ 537us) server_negotiation.cc:407] Connection header received
0811 22:14:59.171649 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.171657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.171856 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.171992 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.172645 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.172650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.172660 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.172671 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.172709 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.172735 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.172868 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.173369 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.173439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":114,"threads_started":1}
I20260811 22:14:59.173861 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.169945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.17.198.2:34741)
0811 22:14:59.170429 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.170441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.170464 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:14:59.170516 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.170521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.170698 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.170862 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.171865 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.171871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.171881 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.171898 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.171945 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.171979 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.172073 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.172671 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.172748 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":167,"threads_started":1}
I20260811 22:14:59.176088 23745 tablet_service.cc:1467] Processing CreateTablet for tablet ce816bc0d8354a96828b101bc47266da (DEFAULT_TABLE table=test-table [id=e68af340282e44498ef1b767efc5ee13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:59.176278 23448 tablet_service.cc:1467] Processing CreateTablet for tablet ce816bc0d8354a96828b101bc47266da (DEFAULT_TABLE table=test-table [id=e68af340282e44498ef1b767efc5ee13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:59.176278 23600 tablet_service.cc:1467] Processing CreateTablet for tablet ce816bc0d8354a96828b101bc47266da (DEFAULT_TABLE table=test-table [id=e68af340282e44498ef1b767efc5ee13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:59.177331 23745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce816bc0d8354a96828b101bc47266da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:59.177342 23448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce816bc0d8354a96828b101bc47266da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:59.177341 23600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce816bc0d8354a96828b101bc47266da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:59.183969 23841 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Bootstrap starting.
I20260811 22:14:59.185354 23842 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Bootstrap starting.
I20260811 22:14:59.185500 23845 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Bootstrap starting.
I20260811 22:14:59.187031 23841 tablet_bootstrap.cc:654] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:59.187543 23842 tablet_bootstrap.cc:654] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:59.187697 23845 tablet_bootstrap.cc:654] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:59.187955 23841 log.cc:824] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:59.188473 23842 log.cc:824] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:59.188473 23845 log.cc:824] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:59.190346 23841 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: No bootstrap required, opened a new log
I20260811 22:14:59.190538 23842 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: No bootstrap required, opened a new log
I20260811 22:14:59.190538 23845 tablet_bootstrap.cc:492] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: No bootstrap required, opened a new log
I20260811 22:14:59.190567 23841 ts_tablet_manager.cc:1397] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:14:59.190716 23842 ts_tablet_manager.cc:1397] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 22:14:59.190716 23845 ts_tablet_manager.cc:1397] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 22:14:59.194371 23841 raft_consensus.cc:348] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.194682 23841 raft_consensus.cc:374] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:59.194741 23841 raft_consensus.cc:729] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8234674f906b40719234b4e32de97e91, State: Initialized, Role: FOLLOWER
I20260811 22:14:59.195228 23841 consensus_queue.cc:260] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.195466 23845 raft_consensus.cc:348] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.195467 23842 raft_consensus.cc:348] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.195760 23845 raft_consensus.cc:374] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:59.195760 23842 raft_consensus.cc:374] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:59.195835 23845 raft_consensus.cc:729] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b8c0a294d4441da351206a9e122638, State: Initialized, Role: FOLLOWER
I20260811 22:14:59.195842 23842 raft_consensus.cc:729] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b33cc74a14465ab9a8da7139b20047, State: Initialized, Role: FOLLOWER
I20260811 22:14:59.196197 23821 heartbeater.cc:503] Master 127.17.198.62:43491 was elected leader, sending a full tablet report...
I20260811 22:14:59.196347 23842 consensus_queue.cc:260] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.196352 23845 consensus_queue.cc:260] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.196465 23841 ts_tablet_manager.cc:1428] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:14:59.197506 23842 ts_tablet_manager.cc:1428] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260811 22:14:59.197506 23845 ts_tablet_manager.cc:1428] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260811 22:14:59.197861 23526 heartbeater.cc:503] Master 127.17.198.62:43491 was elected leader, sending a full tablet report...
I20260811 22:14:59.197857 23676 heartbeater.cc:503] Master 127.17.198.62:43491 was elected leader, sending a full tablet report...
W20260811 22:14:59.249097 23527 tablet.cc:2369] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:59.377825 23822 tablet.cc:2369] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:59.438560 23677 tablet.cc:2369] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:59.446297 23850 raft_consensus.cc:482] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:59.446624 23850 raft_consensus.cc:504] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.447921 23850 leader_election.cc:288] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86b8c0a294d4441da351206a9e122638 (127.17.198.2:34741), a2b33cc74a14465ab9a8da7139b20047 (127.17.198.1:43545)
I20260811 22:14:59.450628 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.448295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43545 (local address 127.17.198.3:35063)
0811 22:14:59.448473 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.448493 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.448666 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.449126 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.449129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.449142 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.449791 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.449793 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.449806 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.449835 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.450284 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.450308 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.450315 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.450415 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.450455 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:14:59.450651 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.448455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:35063 (local address 127.17.198.1:43545)
0811 22:14:59.448608 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.448613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.448631 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:14:59.448894 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.448899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.448945 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.449054 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.450111 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.450118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.450123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.450140 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.450168 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.450181 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.450290 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.450430 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.450470 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:14:59.450805 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.448423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:36453 (local address 127.17.198.2:34741)
0811 22:14:59.448562 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.448567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.449293 (+ 726us) server_negotiation.cc:407] Connection header received
0811 22:14:59.449413 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.449417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.449462 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.449563 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.450375 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.450380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.450386 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.450401 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.450423 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.450434 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.450519 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.450624 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.450668 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:14:59.451196 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.448295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:34741 (local address 127.17.198.3:36453)
0811 22:14:59.449145 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.449159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.449254 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.449647 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.449654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.449670 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:59.450195 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:14:59.450199 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:59.450212 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:14:59.450242 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:59.450519 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:59.450523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:59.450526 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:14:59.450611 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:14:59.450646 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":281,"thread_start_us":270,"threads_started":1}
I20260811 22:14:59.451400 23471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce816bc0d8354a96828b101bc47266da" candidate_uuid: "8234674f906b40719234b4e32de97e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b33cc74a14465ab9a8da7139b20047" is_pre_election: true
I20260811 22:14:59.451824 23471 raft_consensus.cc:2454] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8234674f906b40719234b4e32de97e91 in term 0.
I20260811 22:14:59.451915 23621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce816bc0d8354a96828b101bc47266da" candidate_uuid: "8234674f906b40719234b4e32de97e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b8c0a294d4441da351206a9e122638" is_pre_election: true
I20260811 22:14:59.452179 23621 raft_consensus.cc:2454] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8234674f906b40719234b4e32de97e91 in term 0.
I20260811 22:14:59.452528 23697 leader_election.cc:302] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8234674f906b40719234b4e32de97e91, a2b33cc74a14465ab9a8da7139b20047; no voters:
I20260811 22:14:59.452937 23850 raft_consensus.cc:2781] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:59.453037 23850 raft_consensus.cc:482] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:59.453084 23850 raft_consensus.cc:3037] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:59.454355 23850 raft_consensus.cc:504] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.454746 23850 leader_election.cc:288] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [CANDIDATE]: Term 1 election: Requested vote from peers 86b8c0a294d4441da351206a9e122638 (127.17.198.2:34741), a2b33cc74a14465ab9a8da7139b20047 (127.17.198.1:43545)
I20260811 22:14:59.455211 23471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce816bc0d8354a96828b101bc47266da" candidate_uuid: "8234674f906b40719234b4e32de97e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b33cc74a14465ab9a8da7139b20047"
I20260811 22:14:59.455295 23621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce816bc0d8354a96828b101bc47266da" candidate_uuid: "8234674f906b40719234b4e32de97e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b8c0a294d4441da351206a9e122638"
I20260811 22:14:59.455348 23471 raft_consensus.cc:3037] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:59.455498 23621 raft_consensus.cc:3037] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:59.456617 23471 raft_consensus.cc:2454] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8234674f906b40719234b4e32de97e91 in term 1.
I20260811 22:14:59.456712 23621 raft_consensus.cc:2454] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8234674f906b40719234b4e32de97e91 in term 1.
I20260811 22:14:59.457005 23697 leader_election.cc:302] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8234674f906b40719234b4e32de97e91, a2b33cc74a14465ab9a8da7139b20047; no voters:
I20260811 22:14:59.457190 23850 raft_consensus.cc:2781] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:59.457439 23850 raft_consensus.cc:686] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 1 LEADER]: Becoming Leader. State: Replica: 8234674f906b40719234b4e32de97e91, State: Running, Role: LEADER
I20260811 22:14:59.457916 23850 consensus_queue.cc:237] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } }
I20260811 22:14:59.461747 23313 catalog_manager.cc:5495] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8234674f906b40719234b4e32de97e91 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "8234674f906b40719234b4e32de97e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8234674f906b40719234b4e32de97e91" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260811 22:14:59.477615 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.475655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.17.198.3:46227)
0811 22:14:59.475806 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.475810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.475911 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:14:59.476178 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.476181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.476249 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.476369 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.477166 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.477169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.477172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.477191 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.477210 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.477218 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.477335 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.477422 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.477462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:14:59.484338 23621 raft_consensus.cc:1261] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638 [term 1 FOLLOWER]: Refusing update from remote peer 8234674f906b40719234b4e32de97e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:59.484339 23471 raft_consensus.cc:1261] T ce816bc0d8354a96828b101bc47266da P a2b33cc74a14465ab9a8da7139b20047 [term 1 FOLLOWER]: Refusing update from remote peer 8234674f906b40719234b4e32de97e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:59.485183 23850 consensus_queue.cc:1035] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b33cc74a14465ab9a8da7139b20047" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:59.485421 23855 consensus_queue.cc:1035] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b8c0a294d4441da351206a9e122638" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:59.493901 23859 mvcc.cc:205] Tried to move back new op lower bound from 7317448701879967744 to 7317448701782986752. Current Snapshot: MvccSnapshot[applied={T|T < 7317448701879967744}]
I20260811 22:14:59.500145 23858 mvcc.cc:205] Tried to move back new op lower bound from 7317448701879967744 to 7317448701782986752. Current Snapshot: MvccSnapshot[applied={T|T < 7317448701879967744 or (T in {7317448701879967744})}]
I20260811 22:14:59.503753 23313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:59.503934 23313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:59.506723 23313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f7c26ee5fb14eb59f4abb020d876021: Sending DeleteTablet for 3 replicas of tablet ce816bc0d8354a96828b101bc47266da
I20260811 22:14:59.508021 23600 tablet_service.cc:1514] Processing DeleteTablet for tablet ce816bc0d8354a96828b101bc47266da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:59 UTC) from {username='slave'} at 127.0.0.1:45328
I20260811 22:14:59.508663 23745 tablet_service.cc:1514] Processing DeleteTablet for tablet ce816bc0d8354a96828b101bc47266da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:59 UTC) from {username='slave'} at 127.0.0.1:44924
I20260811 22:14:59.508891 23863 mvcc.cc:205] Tried to move back new op lower bound from 7317448701879967744 to 7317448701782986752. Current Snapshot: MvccSnapshot[applied={T|T < 7317448701879967744}]
I20260811 22:14:59.508745 23448 tablet_service.cc:1514] Processing DeleteTablet for tablet ce816bc0d8354a96828b101bc47266da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:59 UTC) from {username='slave'} at 127.0.0.1:44888
I20260811 22:14:59.509720 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23382
W20260811 22:14:59.521847 23302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:43545 (error 108)
I20260811 22:14:59.522091 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23529
I20260811 22:14:59.522365 23866 tablet_replica.cc:330] T ce816bc0d8354a96828b101bc47266da P 86b8c0a294d4441da351206a9e122638: stopping tablet replica
I20260811 22:14:59.522584 23868 tablet_replica.cc:330] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: stopping tablet replica
W20260811 22:14:59.523612 23301 connection.cc:535] server connection from 127.17.198.1:46667 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:14:59.523669 23868 raft_consensus.cc:2229] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:59.524245 23868 raft_consensus.cc:2258] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:59.524777 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.524324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43545 (local address 127.0.0.1:44902)
0811 22:14:59.524569 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.524650 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 22:14:59.525022 23302 catalog_manager.cc:4584] TS a2b33cc74a14465ab9a8da7139b20047 (127.17.198.1:43545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce816bc0d8354a96828b101bc47266da: Network error: Client connection negotiation failed: client connection to 127.17.198.1:43545: connect: Connection refused (error 111)
I20260811 22:14:59.526327 23868 ts_tablet_manager.cc:1905] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:59.529615 23868 ts_tablet_manager.cc:1918] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:59.529760 23868 log.cc:1196] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/ce816bc0d8354a96828b101bc47266da
I20260811 22:14:59.530145 23868 ts_tablet_manager.cc:1939] T ce816bc0d8354a96828b101bc47266da P 8234674f906b40719234b4e32de97e91: Deleting consensus metadata
I20260811 22:14:59.531675 23302 catalog_manager.cc:4841] TS 8234674f906b40719234b4e32de97e91 (127.17.198.3:46227): tablet ce816bc0d8354a96828b101bc47266da (table test-table [id=e68af340282e44498ef1b767efc5ee13]) successfully deleted
I20260811 22:14:59.532212 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23679
I20260811 22:14:59.534816 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.532412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:34741 (local address 127.0.0.1:45342)
0811 22:14:59.534602 (+ 2190us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.534687 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.2:34741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2035}
I20260811 22:14:59.540350 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23284
2026-08-11T22:14:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1268 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:14:59.566252 18200 mini_oidc.cc:152] Starting JWKS server
I20260811 22:14:59.568248 18200 webserver.cc:460] Webserver started at https://127.0.0.1:33497/ using document root <none> and password file <none>
I20260811 22:14:59.568459 18200 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:14:59.568791 18200 webserver.cc:460] Webserver started at http://127.0.0.1:44211/ using document root <none> and password file <none>
2026-08-11T22:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:59Z Disabled control of system clock
I20260811 22:14:59.581732 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33589
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:33589
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33497/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/testchainca.cert with env {}
W20260811 22:14:59.707898 23877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:59.708293 23877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:59.708379 23877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:59.712532 23877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:59.712612 23877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:59.712628 23877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:59.712646 23877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:59.712661 23877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:14:59.712682 23877 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:14:59.712694 23877 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33497/jwks.json
I20260811 22:14:59.718639 23877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:33589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33497/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:59.719867 23877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:59.721024 23877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:59.730312 23883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:59.730278 23885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:59.730386 23882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:59.730705 23877 server_base.cc:1034] running on GCE node
I20260811 22:14:59.731403 23877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:59.732581 23877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:59.733773 23877 hybrid_clock.cc:648] HybridClock initialized: now 1786486499733751 us; error 42 us; skew 500 ppm
I20260811 22:14:59.736364 23877 webserver.cc:460] Webserver started at http://127.17.198.62:38595/ using document root <none> and password file <none>
I20260811 22:14:59.737138 23877 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:59.737219 23877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:59.737469 23877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:59.740067 23877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "96565265b12a493a9ff71dc83bc4101f"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "c1d049d92965edd79ced41cc8658da86"
server_key_iv: "98acb59dca1765a661081bf747c77026"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.740867 23877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "96565265b12a493a9ff71dc83bc4101f"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "c1d049d92965edd79ced41cc8658da86"
server_key_iv: "98acb59dca1765a661081bf747c77026"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.745517 23877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:59.748787 23891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:59.750617 23877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:59.750828 23877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "96565265b12a493a9ff71dc83bc4101f"
format_stamp: "Formatted at 2026-08-11 22:14:59 on dist-test-slave-77v9"
server_key: "c1d049d92965edd79ced41cc8658da86"
server_key_iv: "98acb59dca1765a661081bf747c77026"
server_key_version: "encryptionkey@0"
I20260811 22:14:59.750993 23877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:59.774489 23877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:59.775317 23877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:59.775557 23877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:59.783583 23877 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:33589
I20260811 22:14:59.783617 23951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:33589 every 8 connection(s)
I20260811 22:14:59.784654 23877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:59.788626 23952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:59.790342 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23877
I20260811 22:14:59.790526 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:14:59.790908 18200 external_mini_cluster.cc:1428] Setting key ebfa63f3034fc7fdb6c76be6ac72f0ac
I20260811 22:14:59.796142 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f: Bootstrap starting.
I20260811 22:14:59.798990 23952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:59.799840 23952 log.cc:824] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:59.802067 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.792541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.17.198.62:33589)
0811 22:14:59.793486 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.793506 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.793549 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:14:59.794417 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.794439 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.794895 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.795274 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.795769 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.797006 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.798034 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.798664 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.798880 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:59.799870 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.799875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.799890 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:59.800445 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.800528 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:59.800561 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.800679 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.800912 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.801220 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":192,"threads_started":1}
I20260811 22:14:59.802601 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f: No bootstrap required, opened a new log
I20260811 22:14:59.805429 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:59.807516 23952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } }
I20260811 22:14:59.807786 23952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:59.807888 23952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96565265b12a493a9ff71dc83bc4101f, State: Initialized, Role: FOLLOWER
I20260811 22:14:59.808410 23952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } }
I20260811 22:14:59.808552 23952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:59.808650 23952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:59.808790 23952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:59.809979 23952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } }
I20260811 22:14:59.810374 23952 leader_election.cc:302] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96565265b12a493a9ff71dc83bc4101f; no voters:
I20260811 22:14:59.810735 23952 leader_election.cc:288] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:59.810884 23958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:59.811136 23958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [term 1 LEADER]: Becoming Leader. State: Replica: 96565265b12a493a9ff71dc83bc4101f, State: Running, Role: LEADER
I20260811 22:14:59.811465 23958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } }
I20260811 22:14:59.812176 23952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:59.813166 23960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96565265b12a493a9ff71dc83bc4101f. Latest consensus state: current_term: 1 leader_uuid: "96565265b12a493a9ff71dc83bc4101f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } } }
I20260811 22:14:59.814186 23960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:59.813578 23959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96565265b12a493a9ff71dc83bc4101f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96565265b12a493a9ff71dc83bc4101f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33589 } } }
I20260811 22:14:59.814605 23959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:59.814654 23972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:59.817441 23972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:59.822865 23972 catalog_manager.cc:1347] Generated new cluster ID: ab4391e7d84342fd8cfd6ddf4a657667
I20260811 22:14:59.822947 23972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:59.829975 23972 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:59.831326 23972 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:59.839246 23972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96565265b12a493a9ff71dc83bc4101f: Generated new TSK 0
I20260811 22:14:59.840229 23972 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:14:59.868094 23984 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:14:59.868299 23984 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:14:59.868960 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.849202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.17.198.62:33589)
0811 22:14:59.849360 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.849367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.849621 (+ 254us) server_negotiation.cc:407] Connection header received
0811 22:14:59.849863 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.849866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.849920 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.850025 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:14:59.850349 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.850908 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.852348 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.852589 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.852680 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:59.853199 (+ 519us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:14:59.868487 (+ 15288us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:14:59.868762 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43466: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:14:59.869048 23956 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43466: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:14:59.870826 23983 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:59.849105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:33589 (local address 127.0.0.1:43466)
0811 22:14:59.849517 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:59.849536 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:59.849690 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:59.850068 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:59.850071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:59.850081 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:14:59.850237 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:59.850245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.851065 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.851069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:59.852202 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:59.852214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.852727 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.852734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:59.852881 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:59.852949 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:14:59.868786 (+ 15837us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:14:59.869127 (+ 341us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.198.62:33589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.198.62:33589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33497/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:14:59.871995 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23877
2026-08-11T22:14:59Z chronyd exiting
I20260811 22:14:59.914606 18200 test_util.cc:175] -----------------------------------------------
I20260811 22:14:59.914718 18200 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486470476166-18200-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (350 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:14:59.915795 18200 mini_oidc.cc:152] Starting JWKS server
I20260811 22:14:59.916719 18200 webserver.cc:460] Webserver started at https://127.0.0.1:34467/ using document root <none> and password file <none>
I20260811 22:14:59.916802 18200 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:14:59.917178 18200 webserver.cc:460] Webserver started at http://127.0.0.1:43773/ using document root <none> and password file <none>
2026-08-11T22:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:59Z Disabled control of system clock
I20260811 22:14:59.927764 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39705
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:39705
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34467/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/testchainca.cert with env {}
W20260811 22:15:00.054579 23991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:00.054910 23991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:00.054955 23991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:00.059222 23991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:00.059314 23991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:00.059329 23991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:00.059346 23991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:00.059362 23991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:15:00.059379 23991 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:15:00.059392 23991 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34467/jwks.json
I20260811 22:15:00.065534 23991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:39705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34467/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:00.066751 23991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:00.067842 23991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:00.077399 23997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.077522 23991 server_base.cc:1034] running on GCE node
W20260811 22:15:00.077399 23998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:00.077435 24000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.078367 23991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:00.079684 23991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:00.080919 23991 hybrid_clock.cc:648] HybridClock initialized: now 1786486500080876 us; error 60 us; skew 500 ppm
I20260811 22:15:00.083098 23991 webserver.cc:460] Webserver started at http://127.17.198.62:36991/ using document root <none> and password file <none>
I20260811 22:15:00.083745 23991 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:00.083849 23991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:00.084096 23991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:00.086534 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "4bcc010133de4606a607bb36ecb56ffe"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "2c12a8e8d8cbc481a3c336b05a635f85"
server_key_iv: "75005ba01062d805e0c23df789c75c2a"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.087198 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "4bcc010133de4606a607bb36ecb56ffe"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "2c12a8e8d8cbc481a3c336b05a635f85"
server_key_iv: "75005ba01062d805e0c23df789c75c2a"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.091747 23991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:15:00.094830 24008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.096521 23991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:00.096740 23991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "4bcc010133de4606a607bb36ecb56ffe"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "2c12a8e8d8cbc481a3c336b05a635f85"
server_key_iv: "75005ba01062d805e0c23df789c75c2a"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.096973 23991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:00.129253 23991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:00.130107 23991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:00.130347 23991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:00.138247 23991 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:39705
I20260811 22:15:00.138260 24067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:39705 every 8 connection(s)
I20260811 22:15:00.139360 23991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:00.142905 24068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:00.146000 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 23991
I20260811 22:15:00.146204 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:00.146524 18200 external_mini_cluster.cc:1428] Setting key 063882c2f2e1eeab89e91c9a704975af
I20260811 22:15:00.149564 24068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe: Bootstrap starting.
I20260811 22:15:00.152168 24068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:00.153224 24068 log.cc:824] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:00.156248 24068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe: No bootstrap required, opened a new log
I20260811 22:15:00.157498 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.148097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.17.198.62:39705)
0811 22:15:00.148903 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.148922 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.148964 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:15:00.149702 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.149750 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.150215 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.150539 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:00.150938 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.152011 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.153369 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.154116 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.154294 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.155317 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:00.155322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:00.155338 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:00.155757 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:00.155820 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:00.155891 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:00.155976 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:00.156266 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 22:15:00.156613 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":171,"threads_started":1}
I20260811 22:15:00.160713 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:15:00.160669 24068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } }
I20260811 22:15:00.161049 24068 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:00.161123 24068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcc010133de4606a607bb36ecb56ffe, State: Initialized, Role: FOLLOWER
I20260811 22:15:00.161671 24068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } }
I20260811 22:15:00.161854 24068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:00.161958 24068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:00.162082 24068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:00.163415 24068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } }
I20260811 22:15:00.163746 24068 leader_election.cc:302] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bcc010133de4606a607bb36ecb56ffe; no voters:
I20260811 22:15:00.164047 24068 leader_election.cc:288] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:00.164186 24073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:00.164498 24073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [term 1 LEADER]: Becoming Leader. State: Replica: 4bcc010133de4606a607bb36ecb56ffe, State: Running, Role: LEADER
I20260811 22:15:00.164875 24073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } }
I20260811 22:15:00.165551 24068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:00.166798 24075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bcc010133de4606a607bb36ecb56ffe. Latest consensus state: current_term: 1 leader_uuid: "4bcc010133de4606a607bb36ecb56ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } } }
I20260811 22:15:00.166795 24074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bcc010133de4606a607bb36ecb56ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcc010133de4606a607bb36ecb56ffe" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39705 } } }
I20260811 22:15:00.167004 24074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.167230 24075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.167481 24083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:00.171535 24083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:00.178151 24083 catalog_manager.cc:1347] Generated new cluster ID: 2bc770dc2acd4fee809a23d1b879e680
I20260811 22:15:00.178249 24083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:00.217069 24083 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:00.218390 24083 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:00.224545 24083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bcc010133de4606a607bb36ecb56ffe: Generated new TSK 0
I20260811 22:15:00.225400 24083 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:15:00.256888 24099 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:15:00.257019 24099 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:15:00.257706 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.239400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.17.198.62:39705)
0811 22:15:00.239559 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.239565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.239739 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:15:00.239909 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.239912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.239980 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.240120 (+ 140us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:15:00.240434 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.241338 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.242393 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.242746 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.242864 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.243229 (+ 365us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:15:00.257211 (+ 13982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34467/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:15:00.257551 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38806: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34467/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:15:00.257786 24071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38806: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34467/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:15:00.257941 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:00.239241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39705 (local address 127.0.0.1:38806)
0811 22:15:00.239642 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:00.239656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:00.239769 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:00.240166 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:00.240169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:00.240178 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:15:00.240292 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.240298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.241509 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.241513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.242253 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.242264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.242908 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.242911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.242980 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.243009 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:15:00.257475 (+ 14466us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34467/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:15:00.257657 (+ 182us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.198.62:39705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34467/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":146,"threads_started":1}
I20260811 22:15:00.258755 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 23991
2026-08-11T22:15:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (386 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:15:00.302290 18200 mini_oidc.cc:152] Starting JWKS server
I20260811 22:15:00.303319 18200 webserver.cc:460] Webserver started at https://127.0.0.1:44675/ using document root <none> and password file <none>
I20260811 22:15:00.303438 18200 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:15:00.303687 18200 webserver.cc:460] Webserver started at http://127.0.0.1:45003/ using document root <none> and password file <none>
2026-08-11T22:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:00Z Disabled control of system clock
I20260811 22:15:00.315413 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37893
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37893
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44675/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:15:00.434124 24106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:00.434487 24106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:00.434573 24106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:00.438925 24106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:00.439059 24106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:00.439090 24106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:00.439121 24106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:00.439157 24106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:15:00.439195 24106 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:15:00.439229 24106 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44675/jwks.json
I20260811 22:15:00.444372 24106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44675/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:00.445672 24106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:00.447165 24106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:00.456702 24116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.456673 24106 server_base.cc:1034] running on GCE node
W20260811 22:15:00.456624 24114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:00.456629 24113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.457808 24106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:00.459246 24106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:00.460522 24106 hybrid_clock.cc:648] HybridClock initialized: now 1786486500460481 us; error 61 us; skew 500 ppm
I20260811 22:15:00.463075 24106 webserver.cc:460] Webserver started at http://127.17.198.62:44633/ using document root <none> and password file <none>
I20260811 22:15:00.463815 24106 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:00.463935 24106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:00.464214 24106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:00.466758 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "9b20b0796e6b4633a48c6c0c5f14d020"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "0e865be7fe72d5d1c7b08adb8bbafc4c"
server_key_iv: "a231180324b483e8f76de93e24306e46"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.467453 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "9b20b0796e6b4633a48c6c0c5f14d020"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "0e865be7fe72d5d1c7b08adb8bbafc4c"
server_key_iv: "a231180324b483e8f76de93e24306e46"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.471948 24106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:15:00.475139 24122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.476789 24106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:15:00.477073 24106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "9b20b0796e6b4633a48c6c0c5f14d020"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "0e865be7fe72d5d1c7b08adb8bbafc4c"
server_key_iv: "a231180324b483e8f76de93e24306e46"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.477228 24106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:00.494544 24106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:00.495347 24106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:00.495574 24106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:00.503795 24185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37893 every 8 connection(s)
I20260811 22:15:00.503809 24106 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37893
I20260811 22:15:00.504995 24106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:00.508778 24186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:00.513154 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24106
I20260811 22:15:00.513397 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:00.513713 18200 external_mini_cluster.cc:1428] Setting key 24ac71cdd458fffbed9aa0f1a190d666
I20260811 22:15:00.516243 24186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020: Bootstrap starting.
I20260811 22:15:00.519219 24186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:00.520408 24186 log.cc:824] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:00.523474 24186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020: No bootstrap required, opened a new log
I20260811 22:15:00.525403 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.515465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.17.198.62:37893)
0811 22:15:00.516188 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.516206 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.516245 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:15:00.517081 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.517122 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.517561 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.517878 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:00.518395 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.519507 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.520984 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.521721 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.521869 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.523069 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:00.523075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:00.523091 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:00.523627 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:00.523710 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:00.523735 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:00.523909 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:00.524100 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 22:15:00.524438 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":184,"threads_started":1}
I20260811 22:15:00.528308 24186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } }
I20260811 22:15:00.528681 24186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:00.528749 24186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b20b0796e6b4633a48c6c0c5f14d020, State: Initialized, Role: FOLLOWER
I20260811 22:15:00.529235 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:15:00.529337 24186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } }
I20260811 22:15:00.529541 24186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:00.529641 24186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:00.529793 24186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:00.531150 24186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } }
I20260811 22:15:00.531597 24186 leader_election.cc:302] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b20b0796e6b4633a48c6c0c5f14d020; no voters:
I20260811 22:15:00.532014 24186 leader_election.cc:288] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:00.532214 24191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:00.532541 24191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [term 1 LEADER]: Becoming Leader. State: Replica: 9b20b0796e6b4633a48c6c0c5f14d020, State: Running, Role: LEADER
I20260811 22:15:00.532948 24191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } }
I20260811 22:15:00.533596 24186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:00.535279 24192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } } }
I20260811 22:15:00.535454 24192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.535770 24194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b20b0796e6b4633a48c6c0c5f14d020. Latest consensus state: current_term: 1 leader_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b20b0796e6b4633a48c6c0c5f14d020" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37893 } } }
I20260811 22:15:00.536013 24194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.536666 24201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:00.539891 24201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:00.546931 24201 catalog_manager.cc:1347] Generated new cluster ID: 1a6ceb4b124b4b8a8c9d9edf82125aab
I20260811 22:15:00.547048 24201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:00.558362 24201 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:00.559489 24201 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:00.565320 24201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b20b0796e6b4633a48c6c0c5f14d020: Generated new TSK 0
I20260811 22:15:00.566175 24201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:00.587806 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.581594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.17.198.62:37893)
0811 22:15:00.581772 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.581779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.582087 (+ 308us) server_negotiation.cc:407] Connection header received
0811 22:15:00.582310 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.582312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.582388 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.582486 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:15:00.582817 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.583472 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.584910 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.585174 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.585270 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.585710 (+ 440us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:15:00.587366 (+ 1656us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44675/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:15:00.587634 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44675/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:15:00.587898 24189 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44675/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:15:00.589701 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:00.581599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37893 (local address 127.0.0.1:38946)
0811 22:15:00.581973 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:00.581989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:00.582121 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:00.582512 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:00.582515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:00.582525 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:15:00.582651 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.582659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.583610 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.583615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.584681 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.584693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.585322 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.585326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.585429 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.585463 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:15:00.587575 (+ 2112us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44675/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:15:00.589500 (+ 1925us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.198.62:37893: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44675/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":1617,"thread_start_us":134,"threads_started":1}
I20260811 22:15:00.590453 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24106
E20260811 22:15:00.603181 24219 webserver.cc:533] Webserver: sslize failed:
E20260811 22:15:00.603276 24219 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T22:15:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (333 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:00Z Disabled control of system clock
I20260811 22:15:00.647411 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35045
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:34757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35045
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/testchainca.cert with env {}
W20260811 22:15:00.773221 24224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:00.773563 24224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:00.773609 24224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:00.778025 24224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:00.778112 24224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:00.778128 24224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:00.778146 24224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:00.778162 24224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:15:00.778179 24224 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:15:00.778193 24224 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:15:00.784111 24224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:34757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:00.785444 24224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:00.786638 24224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:00.797111 24230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.797199 24224 server_base.cc:1034] running on GCE node
W20260811 22:15:00.797093 24229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:00.797113 24232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.798105 24224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:00.799613 24224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:00.800956 24224 hybrid_clock.cc:648] HybridClock initialized: now 1786486500800914 us; error 57 us; skew 500 ppm
I20260811 22:15:00.803126 24224 webserver.cc:460] Webserver started at http://127.17.198.62:39463/ using document root <none> and password file <none>
I20260811 22:15:00.803771 24224 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:00.803879 24224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:00.804098 24224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:00.806562 24224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "d26c76a3d2f843738d3fed8cc1beb116"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "1de6a17b89faa585a80b57a77c1470cb"
server_key_iv: "93d55ab0f52f59de72226202fe4c17f4"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.807190 24224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "d26c76a3d2f843738d3fed8cc1beb116"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "1de6a17b89faa585a80b57a77c1470cb"
server_key_iv: "93d55ab0f52f59de72226202fe4c17f4"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.811270 24224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:15:00.814141 24238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.815671 24224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:00.815848 24224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "d26c76a3d2f843738d3fed8cc1beb116"
format_stamp: "Formatted at 2026-08-11 22:15:00 on dist-test-slave-77v9"
server_key: "1de6a17b89faa585a80b57a77c1470cb"
server_key_iv: "93d55ab0f52f59de72226202fe4c17f4"
server_key_version: "encryptionkey@0"
I20260811 22:15:00.815975 24224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:00.834174 24224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:00.835031 24224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:00.835273 24224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:00.844434 24224 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35045
I20260811 22:15:00.844497 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35045 every 8 connection(s)
I20260811 22:15:00.845847 24224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:00.849304 24297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:00.855989 24297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116: Bootstrap starting.
I20260811 22:15:00.856112 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24224
I20260811 22:15:00.856257 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:00.856773 18200 external_mini_cluster.cc:1428] Setting key 37cc8b51a3d08faf82217d8d563e5ae1
I20260811 22:15:00.859023 24297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:00.859869 24297 log.cc:824] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:00.862378 24297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116: No bootstrap required, opened a new log
I20260811 22:15:00.866343 24297 raft_consensus.cc:348] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } }
I20260811 22:15:00.866693 24297 raft_consensus.cc:374] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:00.866801 24297 raft_consensus.cc:729] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26c76a3d2f843738d3fed8cc1beb116, State: Initialized, Role: FOLLOWER
I20260811 22:15:00.867388 24297 consensus_queue.cc:260] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } }
I20260811 22:15:00.867578 24297 raft_consensus.cc:388] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:00.867657 24297 raft_consensus.cc:482] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:00.867764 24297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:00.868485 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.858294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.17.198.62:35045)
0811 22:15:00.859239 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.859264 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.859312 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:15:00.860156 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.860179 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.860801 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.861308 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:00.861832 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.863006 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.864083 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.864658 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.864941 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.866062 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:00.866068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:00.866089 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:00.866587 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:00.866656 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:00.866679 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:00.866852 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:00.867066 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:15:00.867394 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":166,"threads_started":1}
I20260811 22:15:00.869097 24297 raft_consensus.cc:504] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } }
I20260811 22:15:00.869495 24297 leader_election.cc:302] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d26c76a3d2f843738d3fed8cc1beb116; no voters:
I20260811 22:15:00.869872 24297 leader_election.cc:288] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:00.869958 24302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:00.870165 24302 raft_consensus.cc:686] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [term 1 LEADER]: Becoming Leader. State: Replica: d26c76a3d2f843738d3fed8cc1beb116, State: Running, Role: LEADER
I20260811 22:15:00.870496 24302 consensus_queue.cc:237] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } }
I20260811 22:15:00.871304 24297 sys_catalog.cc:564] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:00.871827 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d26c76a3d2f843738d3fed8cc1beb116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } } }
I20260811 22:15:00.871960 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.872489 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:15:00.872462 24303 sys_catalog.cc:455] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d26c76a3d2f843738d3fed8cc1beb116. Latest consensus state: current_term: 1 leader_uuid: "d26c76a3d2f843738d3fed8cc1beb116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d26c76a3d2f843738d3fed8cc1beb116" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35045 } } }
I20260811 22:15:00.872697 24303 sys_catalog.cc:458] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:00.873163 24307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:00.877070 24307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:00.884192 24307 catalog_manager.cc:1347] Generated new cluster ID: 442e07f8a01143f5a68b3ef71925b66a
I20260811 22:15:00.884310 24307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:00.899525 24307 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:00.900934 24307 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:00.919020 24307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d26c76a3d2f843738d3fed8cc1beb116: Generated new TSK 0
I20260811 22:15:00.920037 24307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:01.389737 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.934638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.17.198.62:35045)
0811 22:15:00.934817 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.934823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.935038 (+ 215us) server_negotiation.cc:407] Connection header received
0811 22:15:00.935226 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.935230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.935301 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.935501 (+ 200us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:15:00.935801 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.936780 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.937990 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.938439 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.938572 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.939013 (+ 441us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:15:01.389251 (+450238us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:15:01.389597 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60274: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:15:01.390002 24300 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60274: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:15:01.390084 24330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:00.934477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35045 (local address 127.0.0.1:60274)
0811 22:15:00.934931 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:00.934945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:00.935064 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:00.935534 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:00.935537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:00.935546 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:15:00.935669 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.935675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.937046 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.937050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.937824 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.937835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.938645 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.938648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.938746 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.938779 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:15:01.389591 (+450812us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:15:01.389794 (+ 203us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.198.62:35045: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":145,"threads_started":1}
I20260811 22:15:01.390957 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24224
2026-08-11T22:15:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (780 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (144 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (141 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:01 dist-test-slave-77v9 krb5kdc[24343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:01 dist-test-slave-77v9 krb5kdc[24343](info): set up 2 sockets
Aug 11 22:15:01 dist-test-slave-77v9 krb5kdc[24343](info): commencing operation
krb5kdc: starting...
W20260811 22:15:02.866550 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:02 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:15:03.026094 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37031
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:03.147037 24358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:03.147440 24358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:03.147542 24358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:03.151952 24358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:03.152101 24358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:03.152140 24358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:03.152181 24358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:03.152218 24358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:03.158197 24358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:03.159519 24358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:03.160657 24358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:03.170094 24364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:03.170158 24363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:03.170260 24358 server_base.cc:1034] running on GCE node
W20260811 22:15:03.170501 24366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:03.171217 24358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:03.172446 24358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:03.173669 24358 hybrid_clock.cc:648] HybridClock initialized: now 1786486503173629 us; error 56 us; skew 500 ppm
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486503, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:03.180421 24358 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:15:03.181584 24358 webserver.cc:460] Webserver started at http://127.17.198.62:43049/ using document root <none> and password file <none>
I20260811 22:15:03.182192 24358 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:03.182263 24358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:03.182451 24358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:03.184777 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "64f1fb0cd2284836b812f53b221d33e7"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "8d3af16e72750ab7d736515556e66542"
server_key_iv: "9f2bcb832d9e8a28a9f4ca446315266b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.185660 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "64f1fb0cd2284836b812f53b221d33e7"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "8d3af16e72750ab7d736515556e66542"
server_key_iv: "9f2bcb832d9e8a28a9f4ca446315266b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.190414 24358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:03.193567 24374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.195164 24358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:03.195377 24358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "64f1fb0cd2284836b812f53b221d33e7"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "8d3af16e72750ab7d736515556e66542"
server_key_iv: "9f2bcb832d9e8a28a9f4ca446315266b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.195539 24358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:03.217574 24358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:03.221550 24358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:03.221810 24358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:03.229820 24358 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37031
I20260811 22:15:03.229861 24429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37031 every 8 connection(s)
I20260811 22:15:03.230858 24358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:03.234087 24431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:03.235280 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24358
I20260811 22:15:03.235442 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:03.235764 18200 external_mini_cluster.cc:1428] Setting key a710db44585f209dfd1c7b7f7ccc4f68
I20260811 22:15:03.241642 24431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7: Bootstrap starting.
I20260811 22:15:03.244401 24431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:03.245385 24431 log.cc:824] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:03.248503 24431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7: No bootstrap required, opened a new log
I20260811 22:15:03.252702 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:03.237489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.17.198.62:37031)
0811 22:15:03.238032 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:03.238050 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:03.238088 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:15:03.238755 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:03.238773 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:03.239151 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:03.239558 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:03.240448 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.241546 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.242498 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.242992 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.243182 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:03.246149 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:03.246155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:03.246172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:03.246202 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:03.248577 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.249295 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.249303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.249310 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.249486 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.249881 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.249885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.249886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.250492 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:03.250747 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:03.251049 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:15:03.251493 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":180,"threads_started":1}
I20260811 22:15:03.254076 24431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } }
I20260811 22:15:03.254427 24431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:03.254498 24431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f1fb0cd2284836b812f53b221d33e7, State: Initialized, Role: FOLLOWER
I20260811 22:15:03.255053 24431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } }
I20260811 22:15:03.255237 24431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:03.255313 24431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:03.255427 24431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:03.256578 24431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } }
I20260811 22:15:03.257012 24431 leader_election.cc:302] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f1fb0cd2284836b812f53b221d33e7; no voters:
I20260811 22:15:03.257387 24431 leader_election.cc:288] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:03.257599 24436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:03.257893 24436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [term 1 LEADER]: Becoming Leader. State: Replica: 64f1fb0cd2284836b812f53b221d33e7, State: Running, Role: LEADER
I20260811 22:15:03.258332 24436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } }
I20260811 22:15:03.258905 24431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:03.260433 24437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f1fb0cd2284836b812f53b221d33e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } } }
I20260811 22:15:03.260473 24438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f1fb0cd2284836b812f53b221d33e7. Latest consensus state: current_term: 1 leader_uuid: "64f1fb0cd2284836b812f53b221d33e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f1fb0cd2284836b812f53b221d33e7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37031 } } }
I20260811 22:15:03.260583 24437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:03.260586 24438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:03.261040 24445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:03.264744 24445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:03.271720 24445 catalog_manager.cc:1347] Generated new cluster ID: 03e75c5b63d845e39966ac41bd65ff6f
I20260811 22:15:03.271855 24445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:03.289994 24445 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:03.291442 24445 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:03.300446 24445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f1fb0cd2284836b812f53b221d33e7: Generated new TSK 0
I20260811 22:15:03.301505 24445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:15:03.346166 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:03.481105 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:03.481606 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:03.481710 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:03.486745 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:03.486894 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:03.487020 24458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:03.493487 24458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:03.494863 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:03.496081 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:03.508291 24466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:03.508286 24468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:03.508556 24458 server_base.cc:1034] running on GCE node
W20260811 22:15:03.508291 24465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:03.509416 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:03.510294 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:03.511488 24458 hybrid_clock.cc:648] HybridClock initialized: now 1786486503511463 us; error 43 us; skew 500 ppm
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486503, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:03.515300 24458 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:15:03.516549 24458 webserver.cc:460] Webserver started at http://127.17.198.1:42733/ using document root <none> and password file <none>
I20260811 22:15:03.517323 24458 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:03.517410 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:03.517675 24458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:03.520501 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "e4002c3070d04a5f8c1feb5b0421f582"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "756bea104a013c48bd00a19fad6bda2f"
server_key_iv: "8cd4ddbdfb63247c9078fa326f05811b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.521341 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "e4002c3070d04a5f8c1feb5b0421f582"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "756bea104a013c48bd00a19fad6bda2f"
server_key_iv: "8cd4ddbdfb63247c9078fa326f05811b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.525843 24458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:03.528998 24475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.530630 24458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:15:03.530853 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "e4002c3070d04a5f8c1feb5b0421f582"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "756bea104a013c48bd00a19fad6bda2f"
server_key_iv: "8cd4ddbdfb63247c9078fa326f05811b"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.531015 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:03.548728 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:03.552664 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:03.552969 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:03.553655 24458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:03.554608 24458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:03.554688 24458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.554765 24458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:03.554816 24458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.565846 24458 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:40495
I20260811 22:15:03.565866 24603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:40495 every 8 connection(s)
I20260811 22:15:03.567000 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:03.576624 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24458
I20260811 22:15:03.576797 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:03.577167 18200 external_mini_cluster.cc:1428] Setting key 5f41c03a602b1662972a8bb58741f005
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486503, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:03.585441 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:03.569387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:54719 (local address 127.17.198.62:37031)
0811 22:15:03.569556 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:03.569560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:03.570558 (+ 998us) server_negotiation.cc:407] Connection header received
0811 22:15:03.572272 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:03.572276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:03.572364 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:03.572489 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:03.574045 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.574652 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.576114 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.576549 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.576701 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:03.580314 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:03.580321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:03.580327 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:03.580368 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:03.582505 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.583355 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.583359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.583360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.583470 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.583944 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.583947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.583949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.584168 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:03.584287 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:03.585139 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 22:15:03.585299 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:15:03.586899 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:03.569734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37031 (local address 127.17.198.1:54719)
0811 22:15:03.570346 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:03.570413 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:03.571981 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:03.572710 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:03.572719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:03.573216 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:03.573796 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:03.573812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.574835 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.574839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:03.575921 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:03.575933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.576915 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.576922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:03.577164 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:03.578073 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:03.578102 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:03.580076 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:03.582685 (+ 2609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:03.582695 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:03.582719 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:03.583203 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:03.583586 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:03.583592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:03.583596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:03.583828 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:03.584300 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:03.584311 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:03.584767 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 22:15:03.585070 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 22:15:03.585520 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":170,"threads_started":1}
I20260811 22:15:03.588554 24604 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37031
I20260811 22:15:03.589007 24604 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:15:03.589648 24604 heartbeater.cc:511] Master 127.17.198.62:37031 requested a full tablet report, sending...
I20260811 22:15:03.591583 24392 ts_manager.cc:194] Registered new tserver with Master: e4002c3070d04a5f8c1feb5b0421f582 (127.17.198.1:40495)
I20260811 22:15:03.593065 24392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:54719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:15:03.612996 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:03.742422 24612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:03.742763 24612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:03.742815 24612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:03.747208 24612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:03.747299 24612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:03.747371 24612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:03.753425 24612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:03.754582 24612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:03.755650 24612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:03.767257 24623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:03.767241 24620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:03.767241 24618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:03.768059 24612 server_base.cc:1034] running on GCE node
I20260811 22:15:03.768546 24612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:03.769420 24612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:03.770634 24612 hybrid_clock.cc:648] HybridClock initialized: now 1786486503770558 us; error 105 us; skew 500 ppm
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486503, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:03.774369 24612 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:15:03.775681 24612 webserver.cc:460] Webserver started at http://127.17.198.2:36519/ using document root <none> and password file <none>
I20260811 22:15:03.776433 24612 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:03.776561 24612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:03.776926 24612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:03.779533 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "f00d75df304e42c196ec72a3d3a0bc9c"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "f56382e4088e7997dcd9a202950cf72e"
server_key_iv: "1556575ae4821d2d922df383f8fe4a36"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.780309 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "f00d75df304e42c196ec72a3d3a0bc9c"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "f56382e4088e7997dcd9a202950cf72e"
server_key_iv: "1556575ae4821d2d922df383f8fe4a36"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.785633 24612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 22:15:03.789299 24631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.791047 24612 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:03.791292 24612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "f00d75df304e42c196ec72a3d3a0bc9c"
format_stamp: "Formatted at 2026-08-11 22:15:03 on dist-test-slave-77v9"
server_key: "f56382e4088e7997dcd9a202950cf72e"
server_key_iv: "1556575ae4821d2d922df383f8fe4a36"
server_key_version: "encryptionkey@0"
I20260811 22:15:03.791491 24612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:03.806466 24612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:03.810556 24612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:03.810743 24612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:03.811354 24612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:03.812353 24612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:03.812407 24612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.812455 24612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:03.812470 24612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:03.823124 24612 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:39445
I20260811 22:15:03.823148 24759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:39445 every 8 connection(s)
I20260811 22:15:03.824189 24612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:03.832445 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24612
I20260811 22:15:03.832559 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:03.832974 18200 external_mini_cluster.cc:1428] Setting key df49a8ce22a453bdf6f38828bf26dd04
Aug 11 22:15:03 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486503, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:03.841849 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:03.826352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:53709 (local address 127.17.198.62:37031)
0811 22:15:03.826538 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:03.826543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:03.827532 (+ 989us) server_negotiation.cc:407] Connection header received
0811 22:15:03.829208 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:03.829212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:03.829291 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:03.829409 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:03.830818 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.831414 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.832874 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.833234 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.833376 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:03.837236 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:03.837240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:03.837242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:03.837280 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:03.839239 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.840029 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.840033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.840037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.840139 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:03.840558 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:03.840562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:03.840564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:03.840780 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:03.840949 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:03.841525 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:15:03.841671 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:15:03.842819 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:03.826654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37031 (local address 127.17.198.2:53709)
0811 22:15:03.827240 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:03.827294 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:03.828943 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:03.829619 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:03.829627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:03.830077 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:03.830632 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:03.830647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.831578 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.831581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:03.832688 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:03.832700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:03.833439 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:03.833444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:03.833698 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:03.834567 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:03.834593 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:03.836986 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:03.839423 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:03.839432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:03.839456 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:03.839787 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:03.840253 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:03.840256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:03.840258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:03.840448 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:03.840948 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:03.840957 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:03.841259 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 22:15:03.841479 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 22:15:03.841872 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":141,"threads_started":1}
I20260811 22:15:03.844290 24760 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37031
I20260811 22:15:03.844607 24760 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:15:03.845352 24760 heartbeater.cc:511] Master 127.17.198.62:37031 requested a full tablet report, sending...
I20260811 22:15:03.846796 24392 ts_manager.cc:194] Registered new tserver with Master: f00d75df304e42c196ec72a3d3a0bc9c (127.17.198.2:39445)
I20260811 22:15:03.847541 24392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:53709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:15:03.870046 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:04.001524 24765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:04.001859 24765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:04.001952 24765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:04.006336 24765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:04.006436 24765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:04.006578 24765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:04.012719 24765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:04.014104 24765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:04.015328 24765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:04.026374 24770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:04.026337 24773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:04.026651 24765 server_base.cc:1034] running on GCE node
W20260811 22:15:04.026786 24771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:04.027371 24765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:04.028410 24765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:04.029659 24765 hybrid_clock.cc:648] HybridClock initialized: now 1786486504029598 us; error 83 us; skew 500 ppm
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.032953 24765 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:15:04.033979 24765 webserver.cc:460] Webserver started at http://127.17.198.3:40525/ using document root <none> and password file <none>
I20260811 22:15:04.034559 24765 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:04.034629 24765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:04.034988 24765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:04.037474 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "4523af8331ec492881462ae420acf8e5"
format_stamp: "Formatted at 2026-08-11 22:15:04 on dist-test-slave-77v9"
server_key: "52b72f2dd6283745cb4185c79b98464f"
server_key_iv: "4c8eed184336b8f215229c46f66b8d4f"
server_key_version: "encryptionkey@0"
I20260811 22:15:04.038146 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "4523af8331ec492881462ae420acf8e5"
format_stamp: "Formatted at 2026-08-11 22:15:04 on dist-test-slave-77v9"
server_key: "52b72f2dd6283745cb4185c79b98464f"
server_key_iv: "4c8eed184336b8f215229c46f66b8d4f"
server_key_version: "encryptionkey@0"
I20260811 22:15:04.042614 24765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:04.046065 24780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:04.048089 24765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:15:04.048339 24765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "4523af8331ec492881462ae420acf8e5"
format_stamp: "Formatted at 2026-08-11 22:15:04 on dist-test-slave-77v9"
server_key: "52b72f2dd6283745cb4185c79b98464f"
server_key_iv: "4c8eed184336b8f215229c46f66b8d4f"
server_key_version: "encryptionkey@0"
I20260811 22:15:04.048512 24765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:04.078732 24765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:04.082691 24765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:04.082960 24765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:04.083699 24765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:04.084700 24765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:04.084784 24765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:04.084884 24765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:04.084930 24765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:04.095486 24765 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:46151
I20260811 22:15:04.095518 24907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:46151 every 8 connection(s)
I20260811 22:15:04.096616 24765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:04.099963 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24765
I20260811 22:15:04.100160 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:04.100644 18200 external_mini_cluster.cc:1428] Setting key 789d0507fc021d6fe16bafedb1b26c65
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:04.114547 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:04.099149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:59527 (local address 127.17.198.62:37031)
0811 22:15:04.099324 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.099328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.100418 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 22:15:04.102120 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.102124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.102188 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.102279 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.103753 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.104288 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.105563 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.105802 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.105931 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:04.109071 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:04.109075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:04.109077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:04.109112 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:04.110824 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.111488 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.111491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.111493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.111651 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.112512 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.112523 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.112525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.113198 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:04.113348 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:04.114066 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 22:15:04.114252 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:15:04.115633 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:04.099591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37031 (local address 127.17.198.3:59527)
0811 22:15:04.100189 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:04.100249 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:04.101848 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:04.102500 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:04.102508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:04.102930 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:04.103591 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:04.103611 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.104437 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.104441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:04.105419 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:04.105430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.105996 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.106000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:04.106194 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:04.106953 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:04.106972 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:04.108712 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:04.110986 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:04.110993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:04.111015 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:04.111339 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:04.111761 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:04.111765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:04.111767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:04.112331 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:04.113388 (+ 1057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:04.113399 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:04.113762 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 22:15:04.113995 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 22:15:04.114524 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":175,"threads_started":1}
I20260811 22:15:04.116966 24908 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37031
I20260811 22:15:04.117257 24908 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:04.117851 24908 heartbeater.cc:511] Master 127.17.198.62:37031 requested a full tablet report, sending...
I20260811 22:15:04.119078 24392 ts_manager.cc:194] Registered new tserver with Master: 4523af8331ec492881462ae420acf8e5 (127.17.198.3:46151)
I20260811 22:15:04.119737 24392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:59527
I20260811 22:15:04.129390 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/test-admin.keytab.
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.151132 18200 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:15:04.154763 24916 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:04.153016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.17.198.1:40495)
0811 22:15:04.153446 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.153456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.153494 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:15:04.153551 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.153557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.153756 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.153894 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.154564 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34126: BlockingRecv error: recv got EOF from 127.0.0.1:34126 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":84,"threads_started":1}
W20260811 22:15:04.154773 18200 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.156097 18200 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.157289 18200 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:15:04.169642 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:04.158380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.17.198.1:40495)
0811 22:15:04.158557 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.158564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.158655 (+ 91us) server_negotiation.cc:407] Connection header received
0811 22:15:04.158818 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.158822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.158876 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.159025 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.159657 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.160524 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.161632 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.162151 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.162263 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:04.165267 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:04.165280 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:04.165296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:04.165325 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:04.167589 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.168141 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.168147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.168149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.168238 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.168540 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.168543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.168549 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.168925 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:04.169104 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:04.169365 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:15:04.169513 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260811 22:15:04.174470 24916 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:04.173190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.17.198.1:40495)
0811 22:15:04.173353 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.173357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.173507 (+ 150us) server_negotiation.cc:407] Connection header received
0811 22:15:04.173673 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.173676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.173726 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.173814 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.174310 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34158: BlockingRecv error: recv got EOF from 127.0.0.1:34158 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:15:04.174693 18200 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.175892 18200 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.177050 18200 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:15:04.188194 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:04.178254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.17.198.1:40495)
0811 22:15:04.178427 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.178431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.178448 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:04.178579 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.178582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.178623 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.178707 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.179342 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.179904 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.180928 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.181178 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.181288 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:04.184126 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:04.184130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:04.184132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:04.184167 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:04.186119 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.186642 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.186645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.186646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.186820 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.187148 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.187150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.187152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.187382 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:04.187527 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:04.187801 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 22:15:04.188000 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 22:15:04.191519 24916 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:04.190473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.17.198.1:40495)
0811 22:15:04.190641 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.190645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.190660 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:04.190832 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.190835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.190881 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.190959 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.191395 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34182: BlockingRecv error: recv got EOF from 127.0.0.1:34182 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:15:04.191636 18200 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486470476166-18200-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.192924 18200 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:04.193998 18200 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:15:04.204245 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:04.195104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.17.198.1:40495)
0811 22:15:04.195229 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:04.195233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:04.195371 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:15:04.195476 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:04.195479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:04.195552 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:04.195671 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:04.196198 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.196747 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.197702 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:04.198021 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:04.198129 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:04.200907 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:04.200910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:04.200912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:04.200941 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:04.202431 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.202924 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.202928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.202929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.203022 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:04.203326 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:04.203331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:04.203334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:04.203534 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:04.203701 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:04.203951 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:15:04.204071 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:15:04.205520 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24458
I20260811 22:15:04.215929 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24612
I20260811 22:15:04.224395 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24765
I20260811 22:15:04.233482 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24358
2026-08-11T22:15:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2557 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24930](info): set up 2 sockets
Aug 11 22:15:04 dist-test-slave-77v9 krb5kdc[24930](info): commencing operation
krb5kdc: starting...
W20260811 22:15:05.385967 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:05 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab.
Entry for principal oryx/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab.
I20260811 22:15:05.540521 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42627
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab
--principal=oryx/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:05.680362 24946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:05.680755 24946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:05.680866 24946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:05.685689 24946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:05.685809 24946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:05.685848 24946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:05.685890 24946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:05.685913 24946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:05.692639 24946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab
--principal=oryx/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:05.693949 24946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:05.695067 24946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:15:05.704906 24946 server_base.cc:1034] running on GCE node
W20260811 22:15:05.704967 24952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:05.704927 24953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:05.704892 24955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:05.705863 24946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:05.707324 24946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:05.708576 24946 hybrid_clock.cc:648] HybridClock initialized: now 1786486505708563 us; error 41 us; skew 500 ppm
Aug 11 22:15:05 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:05.711838 24946 init.cc:377] Logged in from keytab as oryx/127.17.198.62@KRBTEST.COM (short username oryx)
I20260811 22:15:05.712991 24946 webserver.cc:460] Webserver started at http://127.17.198.62:44627/ using document root <none> and password file <none>
I20260811 22:15:05.713580 24946 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:05.713647 24946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:05.713831 24946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:05.716144 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "78efa88636f646e29524b2c9cb4bd772"
format_stamp: "Formatted at 2026-08-11 22:15:05 on dist-test-slave-77v9"
server_key: "ba09a4d68011449a2d17273db0bebb77"
server_key_iv: "7e4dc402ccc9fc4fb73db657419ce5cf"
server_key_version: "encryptionkey@0"
I20260811 22:15:05.716754 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "78efa88636f646e29524b2c9cb4bd772"
format_stamp: "Formatted at 2026-08-11 22:15:05 on dist-test-slave-77v9"
server_key: "ba09a4d68011449a2d17273db0bebb77"
server_key_iv: "7e4dc402ccc9fc4fb73db657419ce5cf"
server_key_version: "encryptionkey@0"
I20260811 22:15:05.721213 24946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:05.725313 24965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:05.727219 24946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:05.727432 24946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "78efa88636f646e29524b2c9cb4bd772"
format_stamp: "Formatted at 2026-08-11 22:15:05 on dist-test-slave-77v9"
server_key: "ba09a4d68011449a2d17273db0bebb77"
server_key_iv: "7e4dc402ccc9fc4fb73db657419ce5cf"
server_key_version: "encryptionkey@0"
I20260811 22:15:05.727607 24946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:05.746338 24946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:05.750439 24946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:05.750643 24946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:05.759335 24946 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:42627
I20260811 22:15:05.759727 25022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:42627 every 8 connection(s)
I20260811 22:15:05.760432 24946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:05.763556 25023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:05.769928 25023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Bootstrap starting.
I20260811 22:15:05.770146 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 24946
I20260811 22:15:05.770331 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:05.770660 18200 external_mini_cluster.cc:1428] Setting key 90238efcaa3b6eb0073d0d179a94915d
I20260811 22:15:05.773088 25023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:05.773955 25023 log.cc:824] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:05.776135 25023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: No bootstrap required, opened a new log
Aug 11 22:15:05 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:05.780369 25023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } }
I20260811 22:15:05.780692 25023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:05.780771 25023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78efa88636f646e29524b2c9cb4bd772, State: Initialized, Role: FOLLOWER
I20260811 22:15:05.781342 25023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } }
I20260811 22:15:05.781533 25023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:05.781618 25023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:05.781751 25023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:05.783097 25023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } }
I20260811 22:15:05.783516 25023 leader_election.cc:302] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78efa88636f646e29524b2c9cb4bd772; no voters:
I20260811 22:15:05.783914 25023 leader_election.cc:288] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:05.784034 25028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:05.784232 25028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 LEADER]: Becoming Leader. State: Replica: 78efa88636f646e29524b2c9cb4bd772, State: Running, Role: LEADER
I20260811 22:15:05.784572 25028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } }
I20260811 22:15:05.785530 25023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:05.786581 25029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } }
I20260811 22:15:05.786759 25029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:05.787163 25038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:05.788046 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:05.772173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.17.198.62:42627)
0811 22:15:05.772719 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:05.772735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:05.772777 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:15:05.773703 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:05.773727 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:05.774146 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:05.774597 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:05.775495 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:05.776602 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:05.777617 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:05.778155 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:05.778372 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:05.781457 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:05.781463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:05.781480 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:05.781543 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:05.783908 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:05.784662 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:05.784670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:05.784677 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:05.784887 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:05.785200 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:05.785205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:05.785206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:05.785727 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:05.785971 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:05.786306 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 22:15:05.786703 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":164,"threads_started":1}
I20260811 22:15:05.790729 25038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:05.791213 25030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78efa88636f646e29524b2c9cb4bd772. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } }
I20260811 22:15:05.791492 25030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:05.797969 25038 catalog_manager.cc:1347] Generated new cluster ID: f644e6748d694942ab3c59635b1cd0f4
I20260811 22:15:05.798085 25038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:05.803555 25038 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:05.804680 25038 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:05.813632 25038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Generated new TSK 0
I20260811 22:15:05.814550 25038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
I20260811 22:15:05.856243 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:05.977739 25051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:05.978068 25051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:05.978116 25051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:05.982529 25051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:05.982623 25051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:05.982686 25051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:05.988687 25051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:05.989995 25051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:05.991132 25051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:06.001646 25058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.001673 25057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.001850 25051 server_base.cc:1034] running on GCE node
W20260811 22:15:06.001688 25060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.003192 25051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.004067 25051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.005287 25051 hybrid_clock.cc:648] HybridClock initialized: now 1786486506005269 us; error 39 us; skew 500 ppm
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.009119 25051 init.cc:377] Logged in from keytab as oryx/127.17.198.1@KRBTEST.COM (short username oryx)
I20260811 22:15:06.010257 25051 webserver.cc:460] Webserver started at http://127.17.198.1:46269/ using document root <none> and password file <none>
I20260811 22:15:06.010957 25051 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.011071 25051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.011338 25051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.013916 25051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "01a6303526d44853b8761ef8c98f3699"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "0c7ab3eb1a03ec5193d2912ea6edef2b"
server_key_iv: "8caa6b67bf48421bca19be13eefc55e4"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.014629 25051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "01a6303526d44853b8761ef8c98f3699"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "0c7ab3eb1a03ec5193d2912ea6edef2b"
server_key_iv: "8caa6b67bf48421bca19be13eefc55e4"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.019418 25051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:06.022451 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.023916 25051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:06.024111 25051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "01a6303526d44853b8761ef8c98f3699"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "0c7ab3eb1a03ec5193d2912ea6edef2b"
server_key_iv: "8caa6b67bf48421bca19be13eefc55e4"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.024261 25051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.061789 25051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.065876 25051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.066135 25051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.066828 25051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:06.067783 25051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:06.067873 25051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.067950 25051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:06.068001 25051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.078383 25051 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:41559
I20260811 22:15:06.078423 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:41559 every 8 connection(s)
I20260811 22:15:06.079392 25051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:06.085731 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25051
I20260811 22:15:06.085904 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:06.086266 18200 external_mini_cluster.cc:1428] Setting key 265099c13029c67bb9f8bb048cc7c501
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:06.096801 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.081533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:42177 (local address 127.17.198.62:42627)
0811 22:15:06.081722 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.081726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.082705 (+ 979us) server_negotiation.cc:407] Connection header received
0811 22:15:06.084297 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.084301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.084355 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.084445 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.086394 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.087261 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.088682 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.089142 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.089282 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.092555 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.092558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.092560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.092588 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.094344 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.095022 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.095026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.095028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.095121 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.095511 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.095518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.095520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.095738 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.095852 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.096423 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.096598 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
WARNING: no policy specified for oryx/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:15:06.097834 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.081936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:42177)
0811 22:15:06.082506 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.082562 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.084036 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.084645 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.084652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.085217 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.086089 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.086116 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.087405 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.087407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.088535 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.088547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.089336 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.089339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.089519 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.090297 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.090327 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.092319 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.094552 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.094559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.094579 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.094899 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.095223 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.095226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.095228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.095395 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.095853 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.095864 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.096159 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.096381 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.096757 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":144,"threads_started":1}
I20260811 22:15:06.099463 25192 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:06.099790 25192 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:06.100443 25192 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.2@KRBTEST.COM" created.
I20260811 22:15:06.102461 24984 ts_manager.cc:194] Registered new tserver with Master: 01a6303526d44853b8761ef8c98f3699 (127.17.198.1:41559)
I20260811 22:15:06.103710 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:42177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
I20260811 22:15:06.121522 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:06.239882 25199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:06.240198 25199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:06.240247 25199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:06.244611 25199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:06.244705 25199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:06.244771 25199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:06.250648 25199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:06.251876 25199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:06.253078 25199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:06.263676 25205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.263684 25207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.263682 25204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.263736 25199 server_base.cc:1034] running on GCE node
I20260811 22:15:06.264516 25199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.265316 25199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.266564 25199 hybrid_clock.cc:648] HybridClock initialized: now 1786486506266552 us; error 36 us; skew 500 ppm
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.270015 25199 init.cc:377] Logged in from keytab as oryx/127.17.198.2@KRBTEST.COM (short username oryx)
I20260811 22:15:06.271032 25199 webserver.cc:460] Webserver started at http://127.17.198.2:41299/ using document root <none> and password file <none>
I20260811 22:15:06.271649 25199 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.271754 25199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.272023 25199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.274314 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "79968fcee6cb448d9856748434cd2600"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "5c323387bef7a19449f6b514bba88150"
server_key_iv: "0492b6eb41d1d966cb3544e0204c2035"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.274935 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "79968fcee6cb448d9856748434cd2600"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "5c323387bef7a19449f6b514bba88150"
server_key_iv: "0492b6eb41d1d966cb3544e0204c2035"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.279142 25199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:06.281842 25215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.283301 25199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:06.283473 25199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "79968fcee6cb448d9856748434cd2600"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "5c323387bef7a19449f6b514bba88150"
server_key_iv: "0492b6eb41d1d966cb3544e0204c2035"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.283612 25199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.294417 25199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.297720 25199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.297972 25199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.298622 25199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:06.299572 25199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:06.299652 25199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.299727 25199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:06.299773 25199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.309152 25199 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:41433
I20260811 22:15:06.309196 25345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:41433 every 8 connection(s)
I20260811 22:15:06.310196 25199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:06.319837 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25199
I20260811 22:15:06.319988 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:06.320305 18200 external_mini_cluster.cc:1428] Setting key 761819ad94dd8bbe63dc9f3e9182ab7a
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:06.325639 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.312198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:41973 (local address 127.17.198.62:42627)
0811 22:15:06.312347 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.312350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.313296 (+ 946us) server_negotiation.cc:407] Connection header received
0811 22:15:06.314910 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.314914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.314975 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.315085 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.316461 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.317088 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.318283 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.318505 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.318616 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.321814 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.321818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.321820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.321853 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.323385 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.323970 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.323973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.323974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.324051 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.324423 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.324428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.324430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.324629 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.324744 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.325333 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.325467 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:15:06.326557 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.312591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:41973)
0811 22:15:06.313140 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.313187 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.314674 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.315271 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.315278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.315701 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.316286 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.316301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.317222 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.317225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.318133 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.318142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.318684 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.318688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.318945 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.319630 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.319654 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.321627 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.323528 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.323535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.323553 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.323839 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.324150 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.324153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.324155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.324312 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.324744 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.324752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.325090 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.325293 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.325664 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":202,"threads_started":1}
I20260811 22:15:06.327859 25346 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:06.328125 25346 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:06.328676 25346 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
I20260811 22:15:06.330024 24984 ts_manager.cc:194] Registered new tserver with Master: 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:06.330626 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:41973
WARNING: no policy specified for oryx/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
I20260811 22:15:06.357121 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:06.488090 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:06.488686 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:06.488937 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:06.493739 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:06.493837 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:06.493911 25352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:06.500257 25352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:06.501631 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:06.502892 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:15:06.514762 25352 server_base.cc:1034] running on GCE node
W20260811 22:15:06.514701 25359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.514707 25361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.514701 25358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.515633 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.516413 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.517650 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786486506517611 us; error 58 us; skew 500 ppm
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.521009 25352 init.cc:377] Logged in from keytab as oryx/127.17.198.3@KRBTEST.COM (short username oryx)
I20260811 22:15:06.522159 25352 webserver.cc:460] Webserver started at http://127.17.198.3:37347/ using document root <none> and password file <none>
I20260811 22:15:06.522781 25352 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.522884 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.523113 25352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.525426 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "285f34404db921cf714eeef182f7035c"
server_key_iv: "d64b14eda8077db9d66689c7de4d0733"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.526084 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "285f34404db921cf714eeef182f7035c"
server_key_iv: "d64b14eda8077db9d66689c7de4d0733"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.530658 25352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:06.534010 25369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.535605 25352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:06.535799 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "285f34404db921cf714eeef182f7035c"
server_key_iv: "d64b14eda8077db9d66689c7de4d0733"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.535952 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.553725 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.557897 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.558172 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.558911 25352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:06.559873 25352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:06.559954 25352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.560030 25352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:06.560076 25352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.571149 25352 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:42213
I20260811 22:15:06.571173 25496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:42213 every 8 connection(s)
I20260811 22:15:06.572225 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:06.577158 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25352
I20260811 22:15:06.577288 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:06.577622 18200 external_mini_cluster.cc:1428] Setting key 02751e6a67930be55b64c4dba8dd2976
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:06.591238 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.574722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:52657 (local address 127.17.198.62:42627)
0811 22:15:06.575027 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.575035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.576080 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 22:15:06.578066 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.578070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.578145 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.578278 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.579982 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.580719 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.582094 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.582348 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.582469 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.585963 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.585967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.585969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.586001 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.588207 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.589147 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.589151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.589152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.589256 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.589755 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.589757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.589759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.590051 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.590157 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.590855 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.591047 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 22:15:06.592624 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.575145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:52657)
0811 22:15:06.575762 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.575817 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.577768 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.578550 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.578561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.579066 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.579774 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.579798 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.580900 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.580903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.581919 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.581930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.582558 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.582561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.582737 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.583530 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.583550 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.585571 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.588405 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.588414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.588432 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.588967 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.589394 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.589398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.589400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.589620 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.590212 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.590222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.590562 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.590758 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.591335 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":195,"threads_started":1}
I20260811 22:15:06.593969 25497 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:06.594249 25497 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:06.594844 25497 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
I20260811 22:15:06.596222 24984 ts_manager.cc:194] Registered new tserver with Master: b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213)
I20260811 22:15:06.596961 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:52657
I20260811 22:15:06.606137 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.198.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.61.keytab.
Entry for principal oryx/127.17.198.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.61.keytab.
I20260811 22:15:06.647210 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:45713
--webserver_interface=127.17.198.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.61.keytab
--principal=oryx/127.17.198.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42627,127.17.198.61:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:06.773029 25504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:06.773412 25504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:06.773519 25504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:06.777916 25504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:06.778031 25504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:06.778071 25504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:06.778133 25504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:06.778172 25504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:06.784183 25504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42627,127.17.198.61:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.61.keytab
--principal=oryx/127.17.198.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.198.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
W20260811 22:15:06.784515 25504 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.198.62:42627,127.17.198.61:45713'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:15:06.785589 25504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:06.786839 25504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:06.796150 25514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.796150 25512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.796172 25511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.796680 25504 server_base.cc:1034] running on GCE node
I20260811 22:15:06.797487 25504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.798735 25504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.799962 25504 hybrid_clock.cc:648] HybridClock initialized: now 1786486506799929 us; error 48 us; skew 500 ppm
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.803033 25504 init.cc:377] Logged in from keytab as oryx/127.17.198.61@KRBTEST.COM (short username oryx)
I20260811 22:15:06.804057 25504 webserver.cc:460] Webserver started at http://127.17.198.61:46163/ using document root <none> and password file <none>
I20260811 22:15:06.804675 25504 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.804744 25504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.805038 25504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.807286 25504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/instance:
uuid: "50e69706d13b44fab124875994d52d40"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "a696dfd6d96ebdb69be5bfa0a9a0209b"
server_key_iv: "b77a84457c0c9e90d73682476f0006cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.807920 25504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal/instance:
uuid: "50e69706d13b44fab124875994d52d40"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "a696dfd6d96ebdb69be5bfa0a9a0209b"
server_key_iv: "b77a84457c0c9e90d73682476f0006cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.812197 25504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:15:06.815001 25521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.816370 25504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:06.816519 25504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "50e69706d13b44fab124875994d52d40"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "a696dfd6d96ebdb69be5bfa0a9a0209b"
server_key_iv: "b77a84457c0c9e90d73682476f0006cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.816645 25504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.836004 25504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.839609 25504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.839843 25504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.847608 25504 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.61:45713
I20260811 22:15:06.847949 25575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.61:45713 every 8 connection(s)
I20260811 22:15:06.848712 25504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
I20260811 22:15:06.851692 25576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:06.855756 25576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:15:06.856467 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25504
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.61@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
I20260811 22:15:06.871443 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.857997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.17.198.62:42627)
0811 22:15:06.858164 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.858168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.858744 (+ 576us) server_negotiation.cc:407] Connection header received
0811 22:15:06.859529 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.859532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.859581 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.859677 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.861024 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.861818 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.863365 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.863710 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.863822 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.866928 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.866957 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.866963 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.866998 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.868700 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.869432 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.869435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.869436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.869536 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.869897 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.869900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.869902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.870254 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.870360 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.871058 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.871220 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:15:06.872488 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.858073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36644)
0811 22:15:06.858576 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.858627 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.859314 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.859824 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.859831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.860403 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.860777 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.860790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.862001 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.862004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.863239 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.863251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.863862 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.863865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.864056 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.864966 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.864990 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.866750 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.868859 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.868867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.868890 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.869316 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.869625 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.869627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.869628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.869774 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.870365 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.870371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.870729 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.870980 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.871376 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":49,"thread_start_us":146,"threads_started":1}
I20260811 22:15:06.872488 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.858074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.17.198.61:45713)
0811 22:15:06.858534 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.858551 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.858584 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:15:06.859298 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.859315 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.859577 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.859755 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.860730 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.862103 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.863334 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.863914 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.864050 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.867422 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.867430 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.867451 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.867472 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.869326 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.870113 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.870118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.870119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.870185 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.870485 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.870488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.870490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.870777 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.870990 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.871256 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.871723 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":323,"thread_start_us":169,"threads_started":1}
I20260811 22:15:06.874215 25576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.61@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
I20260811 22:15:06.883530 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.874722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37340)
0811 22:15:06.874849 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.874865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.875009 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.875233 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.875235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.875441 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.875544 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.875550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.876238 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.876241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.876752 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.876759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.877933 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.877936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.878012 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.878504 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.878517 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.880298 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.882189 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.882192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.882193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.882412 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.882661 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.882664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.882665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.882739 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.883098 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.883100 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.883177 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.883264 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.883377 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:15:06.883698 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.874882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.17.198.61:45713)
0811 22:15:06.874983 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.874986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.874998 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:15:06.875092 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.875094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.875126 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.875190 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.875643 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.876137 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.876928 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.877776 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.877867 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.880460 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.880464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.880466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.880488 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.882077 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.882504 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.882507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.882508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.882581 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.882816 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.882819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.882820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.883012 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.883092 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.883394 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.883519 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:15:06.887229 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Bootstrap starting.
I20260811 22:15:06.890108 25576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:06.890836 25576 log.cc:824] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:06.893008 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: No bootstrap required, opened a new log
I20260811 22:15:06.896008 25576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } }
I20260811 22:15:06.896220 25576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:06.896252 25576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e69706d13b44fab124875994d52d40, State: Initialized, Role: FOLLOWER
I20260811 22:15:06.896632 25576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } }
I20260811 22:15:06.897169 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } } }
I20260811 22:15:06.897310 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:06.897922 25576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:15:06.902736 25594 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:15:06.902848 25594 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:15:06.905507 25504 master_runner.cc:428] Detected that this master 50e69706d13b44fab124875994d52d40 is joining an existing cluster
I20260811 22:15:06.905593 25504 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.198.61:45713
I20260811 22:15:06.917276 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.907835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.17.198.61:45713)
0811 22:15:06.907979 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.907984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.908462 (+ 478us) server_negotiation.cc:407] Connection header received
0811 22:15:06.908555 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.908558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.908607 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.908698 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.910531 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.911377 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.912414 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.912757 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.912933 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.914051 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.914059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.914061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.914079 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.915404 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.915812 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.915818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.915822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.915899 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.916215 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.916221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.916224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.916489 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.916573 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.916923 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.917058 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":4224}
I20260811 22:15:06.918202 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.908081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37350)
0811 22:15:06.908356 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.908367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.908476 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.908729 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.908733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.909110 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.909392 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.909398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.911491 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.911494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.912273 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.912286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.912964 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.912969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.913067 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.913567 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.913580 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.913919 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.915513 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.915519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.915524 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.915719 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.915988 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.915993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.915995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.916099 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.916637 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.916640 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.916717 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.917835 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.917999 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":60,"thread_start_us":123,"threads_started":1}
I20260811 22:15:06.918273 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.907784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.17.198.62:42627)
0811 22:15:06.908000 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.908004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.908294 (+ 290us) server_negotiation.cc:407] Connection header received
0811 22:15:06.908521 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.908527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.908590 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.908635 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.909529 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.910380 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.911333 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.911671 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.911782 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.913720 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.913724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.913727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.913758 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.916094 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.916651 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.916655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.916657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.916749 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.917188 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.917191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.917193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.917431 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.917510 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.917961 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.918105 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 22:15:06.918740 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.907731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36660)
0811 22:15:06.908189 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.908202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.908387 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.908803 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.908807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.909141 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.909385 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.909395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.910511 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.910518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.911199 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.911208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.911812 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.911816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.911929 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.912961 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.912977 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.913559 (+ 582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.916222 (+ 2663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.916225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.916227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.916529 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.916987 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.916990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.916992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.917082 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.917588 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.917592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.917682 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.918416 (+ 734us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.918564 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":49,"spinlock_wait_cycles":4608,"thread_start_us":161,"threads_started":1}
Aug 11 22:15:06 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.62@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
WARNING: no policy specified for oryx/127.17.198.60@KRBTEST.COM; defaulting to no policy
I20260811 22:15:06.941229 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.927231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37356)
0811 22:15:06.927684 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.927747 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.927892 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.928479 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.928487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.929084 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.929277 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.929287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.930146 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.930150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.930847 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.930857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.932950 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.932955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.933105 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.933902 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.933920 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.936061 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.938590 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.938606 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.938612 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.939082 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.939463 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.939466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.939467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.939660 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.940325 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.940334 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.940487 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.940692 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.941017 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":145,"threads_started":1}
I20260811 22:15:06.941371 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.928023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.17.198.61:45713)
0811 22:15:06.928171 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.928175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.928188 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:15:06.928254 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.928257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.928308 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.928391 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.929449 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.930032 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.930995 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.931371 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.931480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.936285 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.936290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.936293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.936329 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.938398 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.939218 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.939222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.939224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.939336 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.939769 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.939773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.939774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.940146 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.940252 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.941052 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.941176 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.60@KRBTEST.COM" created.
I20260811 22:15:06.942997 24984 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.198.61:45713: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "50e69706d13b44fab124875994d52d40"
member_type: NON_VOTER
last_known_addr {
host: "127.17.198.61"
port: 45713
}
attrs {
promote: true
}
}
dest_uuid: "78efa88636f646e29524b2c9cb4bd772"
cas_config_opid_index: -1
I20260811 22:15:06.943748 24984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } }
I20260811 22:15:06.946128 25608 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 50e69706d13b44fab124875994d52d40 (127.17.198.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } }
I20260811 22:15:06.946664 25549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:06.948086 25608 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.198.61:45713
I20260811 22:15:06.948948 25608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } }
I20260811 22:15:06.949071 25608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:06.949419 25549 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:15:06.950055 25504 master.cc:537] Master@127.17.198.61:45713 shutting down...
I20260811 22:15:06.950356 25609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:15:06.953436 24970 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 22:15:06.955332 24970 consensus_peers.cc:487] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 -> Peer 50e69706d13b44fab124875994d52d40 (127.17.198.61:45713): Couldn't send request to peer 50e69706d13b44fab124875994d52d40. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:15:06.956116 25504 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:06.956465 25504 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:06.956598 25504 tablet_replica.cc:330] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.60.keytab.
Entry for principal oryx/127.17.198.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.60.keytab.
I20260811 22:15:06.966676 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:32911
--webserver_interface=127.17.198.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.60.keytab
--principal=oryx/127.17.198.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:06.972323 25504 master.cc:552] Master@127.17.198.61:45713 shutdown complete.
I20260811 22:15:06.972412 25504 master_runner.cc:305] Clearing existing system tablet
I20260811 22:15:06.972687 25504 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.972751 25504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.974303 25504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:06.975519 25614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.975792 25504 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.975961 25504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:06.976022 25504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "50e69706d13b44fab124875994d52d40"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "a696dfd6d96ebdb69be5bfa0a9a0209b"
server_key_iv: "b77a84457c0c9e90d73682476f0006cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.978533 25504 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:06.981335 25504 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:15:06.981482 25504 log.cc:1196] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:15:06.981783 25504 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Deleting consensus metadata
I20260811 22:15:06.981967 25504 master_runner.cc:315] Copying system tablet from 127.17.198.62:42627
I20260811 22:15:06.983283 25504 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: tablet copy: Beginning tablet copy session from remote peer at address 127.17.198.62:42627
I20260811 22:15:06.994288 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.983938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36666)
0811 22:15:06.984278 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.984296 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.984441 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.984761 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.984764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.985056 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.985249 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.985258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.986398 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.986402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.987599 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.987617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.988294 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.988299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.988413 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.989211 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.989229 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.989754 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.992322 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.992325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.992327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.992599 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.993025 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.993029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.993031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.993130 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.993646 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.993650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.993762 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.993892 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.994056 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":103,"threads_started":1}
I20260811 22:15:06.994321 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.984086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.17.198.62:42627)
0811 22:15:06.984224 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.984229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.984401 (+ 172us) server_negotiation.cc:407] Connection header received
0811 22:15:06.984552 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.984556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.984610 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.984720 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.985403 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.986252 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.987758 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.988130 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.988246 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.989947 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.989951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.989954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.989986 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.992162 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.992724 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.992728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.992730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.992911 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.993241 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.993249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.993251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.993537 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.993649 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.993973 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.994117 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:15:06.995337 25006 tablet_copy_service.cc:140] P 78efa88636f646e29524b2c9cb4bd772: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 50e69706d13b44fab124875994d52d40 ({username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:36666)
I20260811 22:15:06.995549 25006 tablet_copy_service.cc:161] P 78efa88636f646e29524b2c9cb4bd772: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 50e69706d13b44fab124875994d52d40 at {username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:36666: session id = 50e69706d13b44fab124875994d52d40-00000000000000000000000000000000
I20260811 22:15:06.997474 25006 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:15:06.999191 25504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:15:07.001895 25504 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: tablet copy: Starting download of 0 data blocks...
I20260811 22:15:07.002110 25504 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: tablet copy: Starting download of 1 WAL segments...
I20260811 22:15:07.004246 25504 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:15:07.006728 25006 tablet_copy_service.cc:342] P 78efa88636f646e29524b2c9cb4bd772: Request end of tablet copy session 50e69706d13b44fab124875994d52d40-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:36666
I20260811 22:15:07.006922 25006 tablet_copy_service.cc:433] P 78efa88636f646e29524b2c9cb4bd772: ending tablet copy session 50e69706d13b44fab124875994d52d40-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 50e69706d13b44fab124875994d52d40
I20260811 22:15:07.016481 25504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.018922 25621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.018922 25622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.019237 25624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.019760 25504 server_base.cc:1034] running on GCE node
I20260811 22:15:07.019943 25504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.020327 25504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.023638 25504 hybrid_clock.cc:648] HybridClock initialized: now 1786486507022977 us; error 679 us; skew 500 ppm
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.025705 25504 init.cc:377] Logged in from keytab as oryx/127.17.198.61@KRBTEST.COM (short username oryx)
I20260811 22:15:07.026294 25504 webserver.cc:460] Webserver started at http://127.17.198.61:33241/ using document root <none> and password file <none>
I20260811 22:15:07.026535 25504 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.026583 25504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.028060 25504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:15:07.029253 25634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.029553 25504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:07.029690 25504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "50e69706d13b44fab124875994d52d40"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "a696dfd6d96ebdb69be5bfa0a9a0209b"
server_key_iv: "b77a84457c0c9e90d73682476f0006cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.029961 25504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.041695 25504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.044410 25504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.050014 25504 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.61:45713
I20260811 22:15:07.050068 25694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.61:45713 every 8 connection(s)
I20260811 22:15:07.050873 25504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
I20260811 22:15:07.053165 25695 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:15:07.054509 25695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Bootstrap starting.
I20260811 22:15:07.067163 25695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:07.067575 25695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Bootstrap complete.
I20260811 22:15:07.068284 25695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } }
I20260811 22:15:07.068441 25695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50e69706d13b44fab124875994d52d40, State: Initialized, Role: LEARNER
I20260811 22:15:07.068549 25695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } }
I20260811 22:15:07.069105 25695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:07.069257 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } }
I20260811 22:15:07.069553 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:15:07.073046 25710 catalog_manager.cc:1259] Loaded cluster ID: f644e6748d694942ab3c59635b1cd0f4
I20260811 22:15:07.073145 25710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: loading cluster ID for follower catalog manager: success
I20260811 22:15:07.075611 25710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: acquiring CA information for follower catalog manager: success
I20260811 22:15:07.077502 25710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:15:07.106626 25192 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
W20260811 22:15:07.132094 25612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:07.132496 25612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:07.132597 25612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:07.137110 25612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:07.137221 25612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:07.137256 25612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:07.137295 25612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:07.137326 25612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:07.143414 25612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.60.keytab
--principal=oryx/127.17.198.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:32911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.198.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:07.144694 25612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:07.145848 25612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.155352 25715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.155352 25718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.155354 25716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.155750 25612 server_base.cc:1034] running on GCE node
I20260811 22:15:07.156636 25612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.158080 25612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.159291 25612 hybrid_clock.cc:648] HybridClock initialized: now 1786486507159264 us; error 40 us; skew 500 ppm
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.163103 25612 init.cc:377] Logged in from keytab as oryx/127.17.198.60@KRBTEST.COM (short username oryx)
I20260811 22:15:07.164228 25612 webserver.cc:460] Webserver started at http://127.17.198.60:37541/ using document root <none> and password file <none>
I20260811 22:15:07.164921 25612 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.165004 25612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.165199 25612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:07.167572 25612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/instance:
uuid: "b95c531fa328449d8a11730a56385acc"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-77v9"
server_key: "3e5575da025b9a2d1f05cb50c7f50f5d"
server_key_iv: "f351680bdfe088414aa3937dce1eb30e"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.168215 25612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal/instance:
uuid: "b95c531fa328449d8a11730a56385acc"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-77v9"
server_key: "3e5575da025b9a2d1f05cb50c7f50f5d"
server_key_iv: "f351680bdfe088414aa3937dce1eb30e"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.173292 25612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:15:07.176335 25728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.178180 25612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:07.178402 25612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "b95c531fa328449d8a11730a56385acc"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-77v9"
server_key: "3e5575da025b9a2d1f05cb50c7f50f5d"
server_key_iv: "f351680bdfe088414aa3937dce1eb30e"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.178611 25612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.202951 25612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.206910 25612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:07.207165 25612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.216140 25612 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.60:32911
I20260811 22:15:07.216395 25787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.60:32911 every 8 connection(s)
I20260811 22:15:07.217450 25612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
I20260811 22:15:07.220631 25788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:07.224783 25788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:15:07.227198 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25612
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.60@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
I20260811 22:15:07.239761 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.227125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.17.198.62:42627)
0811 22:15:07.227306 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.227311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.228104 (+ 793us) server_negotiation.cc:407] Connection header received
0811 22:15:07.229338 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.229341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.229402 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.229485 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.230485 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.231056 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.232329 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.232630 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.232747 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.235757 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.235761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.235763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.235791 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.237391 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.238044 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.238048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.238050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.238141 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.238515 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.238519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.238524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.238729 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.238837 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.239434 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.239573 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:15:07.240677 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.228036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.17.198.60:32911)
0811 22:15:07.228591 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.228603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.228642 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:15:07.229141 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.229167 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.229374 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.229575 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.230523 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.231532 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.232447 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.232897 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.233009 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.236006 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.236018 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.236031 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.236083 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.237974 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.238449 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.238455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.238456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.238551 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.238865 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.238869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.238871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.239145 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.239353 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.239974 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.240124 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":167,"threads_started":1}
I20260811 22:15:07.240687 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.227422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36674)
0811 22:15:07.227945 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.228000 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.229126 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.229552 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.229558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.229977 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.230301 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.230316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.231225 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.231227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.232197 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.232206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.232790 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.232794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.232985 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.233581 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.233597 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.235558 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.237524 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.237535 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.237547 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.237933 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.238239 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.238243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.238245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.238404 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.238836 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.238844 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.239129 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.239315 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.239721 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":154,"threads_started":1}
I20260811 22:15:07.242591 25788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.60@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
I20260811 22:15:07.253127 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.243411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37366)
0811 22:15:07.243546 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.243561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.243644 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.244131 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.244133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.244288 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.244415 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.244420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.245504 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.245506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.246146 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.246155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.246699 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.246702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.246805 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.247465 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.247479 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.249213 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.251326 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.251329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.251331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.251572 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.251989 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.251992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.251993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.252092 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.252555 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.252557 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.252654 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.252744 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.252992 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:15:07.253356 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.243543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37366 (local address 127.17.198.61:45713)
0811 22:15:07.243890 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.243894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.243909 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:07.243951 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.243953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.243994 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.244081 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.244560 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.245380 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.246295 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.246549 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.246651 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.249393 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.249396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.249398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.249426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.251188 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.251676 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.251679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.251680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.251839 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.252220 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.252223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.252225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.252440 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.252542 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.253042 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.253175 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260811 22:15:07.254218 25788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.60@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
I20260811 22:15:07.263685 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.254778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38130)
0811 22:15:07.254968 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.254985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.255065 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.255341 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.255344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.255505 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.255607 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.255612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.256585 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.256588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.257303 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.257312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.257762 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.257765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.257823 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.258507 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.258520 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.260048 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.262223 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.262226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.262227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.262464 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.262761 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.262764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.262765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.262852 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.263246 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.263248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.263333 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.263435 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.263551 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:15:07.263787 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.254941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.17.198.60:32911)
0811 22:15:07.255112 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.255116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.255130 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:15:07.255168 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.255170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.255207 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.255287 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.255775 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.256445 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.257408 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.257661 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.257751 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.260239 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.260243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.260245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.260269 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.262099 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.262567 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.262570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.262572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.262658 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.262952 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.262955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.262956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.263158 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.263244 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.263516 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.263627 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:15:07.267781 25788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Bootstrap starting.
I20260811 22:15:07.270399 25788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:07.271219 25788 log.cc:824] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:07.273613 25788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: No bootstrap required, opened a new log
I20260811 22:15:07.277159 25788 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } }
I20260811 22:15:07.277426 25788 raft_consensus.cc:374] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:07.277506 25788 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95c531fa328449d8a11730a56385acc, State: Initialized, Role: FOLLOWER
I20260811 22:15:07.278077 25788 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } }
I20260811 22:15:07.278692 25794 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } } }
I20260811 22:15:07.278859 25794 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.279642 25788 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:15:07.285064 25807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:15:07.285177 25807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:15:07.289685 25612 master_runner.cc:428] Detected that this master b95c531fa328449d8a11730a56385acc is joining an existing cluster
I20260811 22:15:07.289844 25612 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.198.60:32911
I20260811 22:15:07.294800 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25051
I20260811 22:15:07.309183 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:41559
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:42627
--webserver_port=46269
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:07.323436 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.17.198.62:42627)
0811 22:15:07.293250 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.293256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.293883 (+ 627us) server_negotiation.cc:407] Connection header received
0811 22:15:07.294051 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.294057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.294116 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.294199 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.294938 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.295759 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.299129 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.299588 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.299699 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.301447 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.301451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.301454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.301483 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.303386 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.321714 (+ 18328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.321720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.321722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.321859 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.322223 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.322227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.322229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.322499 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.322593 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.323031 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.323225 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:15:07.324267 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36676)
0811 22:15:07.293764 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.293795 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.293927 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.294329 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.294333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.294627 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.294788 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.294797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.295917 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.295920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.298904 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.298925 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.299925 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.299929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.300032 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.300671 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.300687 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.301253 (+ 566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.303741 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.303745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.303748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.321501 (+ 17753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.321999 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.322002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.322004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.322106 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.322655 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.322658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.322774 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.323861 (+ 1087us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.324057 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260811 22:15:07.328827 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38140)
0811 22:15:07.294034 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.294048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.294165 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.294526 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.294531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.294808 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.294976 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.294988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.296371 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.296377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.302259 (+ 5882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.302289 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.303176 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.303180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.303370 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.303870 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.303893 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.321657 (+ 17764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.326887 (+ 5230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.326893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.326898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.327164 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.327460 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.327463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.327466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.327554 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.328233 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.328235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.328333 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.328435 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.328590 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":4634,"thread_start_us":129,"threads_started":1}
I20260811 22:15:07.329672 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.17.198.60:32911)
0811 22:15:07.293284 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.293290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.294117 (+ 827us) server_negotiation.cc:407] Connection header received
0811 22:15:07.294255 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.294257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.294320 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.294373 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.295111 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.296194 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.302435 (+ 6241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.302885 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.302994 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.322579 (+ 19585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.322586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.322592 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.322630 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.326746 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.327284 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.327288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.327290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.327368 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.327815 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.327818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.327820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.328095 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.328196 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.329387 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.329510 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:15:07.330106 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37376)
0811 22:15:07.305106 (+ 11519us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.305123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.305234 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.305584 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.305587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.305818 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.305962 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.305969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.307315 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.307319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.308387 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.308401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.309340 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.309344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.309443 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.310064 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.310081 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.322365 (+ 12284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.325977 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.325983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.325989 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.326322 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.327647 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.327650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.327652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.327723 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.329496 (+ 1773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.329502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.329604 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.329714 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.329925 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11428,"thread_start_us":106,"threads_started":1}
I20260811 22:15:07.330788 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.293702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.17.198.61:45713)
0811 22:15:07.293842 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.293848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.305227 (+ 11379us) server_negotiation.cc:407] Connection header received
0811 22:15:07.305342 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.305349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.305429 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.305534 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.306111 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.307169 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.308604 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.309019 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.309139 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.323050 (+ 13911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.323059 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.323065 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.323108 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.325802 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.327085 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.327090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.327092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.327498 (+ 406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.329032 (+ 1534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.329036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.329038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.329326 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.329438 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.330463 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.330622 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:15:07.333057 25346 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.62@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
I20260811 22:15:07.340648 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.328877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38156)
0811 22:15:07.329127 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.329141 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.329217 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.330201 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.330204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.330420 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.330542 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.330551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.331803 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.331807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.332726 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.332738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.333769 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.333773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.333896 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.334508 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.334524 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.336302 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.338749 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.338753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.338755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.339055 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.339433 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.339438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.339440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.339547 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.340036 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.340040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.340162 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.340279 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.340455 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 22:15:07.341104 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.329370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.17.198.60:32911)
0811 22:15:07.329828 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.329832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.329844 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:15:07.329915 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.329918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.329969 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.330044 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.330697 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.331657 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.333286 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.333634 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.333757 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.336500 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.336505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.336509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.336541 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.338594 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.339205 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.339209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.339212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.339319 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.339662 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.339666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.339668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.339926 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.340035 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.340363 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.340499 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":110,"threads_started":1}
I20260811 22:15:07.342253 24984 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.198.60:32911: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b95c531fa328449d8a11730a56385acc"
member_type: NON_VOTER
last_known_addr {
host: "127.17.198.60"
port: 32911
}
attrs {
promote: true
}
}
dest_uuid: "78efa88636f646e29524b2c9cb4bd772"
cas_config_opid_index: 5
I20260811 22:15:07.342975 24984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } }
I20260811 22:15:07.344129 25609 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b95c531fa328449d8a11730a56385acc (127.17.198.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.345801 25609 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.198.60:32911
I20260811 22:15:07.345996 25609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.346099 25609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:07.347122 25612 master.cc:537] Master@127.17.198.60:32911 shutting down...
I20260811 22:15:07.347595 25757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:07.349210 25757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:15:07.350047 25608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:15:07.353714 24969 consensus_peers.cc:487] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 -> Peer b95c531fa328449d8a11730a56385acc (127.17.198.60:32911): Couldn't send request to peer b95c531fa328449d8a11730a56385acc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:15:07.356002 25612 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:07.356365 25612 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:07.356474 25612 tablet_replica.cc:330] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: stopping tablet replica
I20260811 22:15:07.360280 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.346608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.17.198.61:45713)
0811 22:15:07.346747 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.346752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.354800 (+ 8048us) server_negotiation.cc:407] Connection header received
0811 22:15:07.355038 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.355041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.355110 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.355177 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:07.355771 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.356600 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.358363 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.359083 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.359173 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.359345 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.360027 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.360130 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:15:07.360697 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.346466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:37390)
0811 22:15:07.354701 (+ 8235us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.354720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.354905 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.355349 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.355352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.355370 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:07.355608 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.355620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.356760 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.356763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.358171 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.358185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.359695 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.359699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.359809 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.359890 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.360460 (+ 570us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.360564 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8129}
I20260811 22:15:07.361665 25666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 LEARNER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:15:07.362224 25608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:15:07.364253 25666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b95c531fa328449d8a11730a56385acc (127.17.198.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.366900 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78efa88636f646e29524b2c9cb4bd772. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.367067 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:15:07.367163 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: NON_VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: true } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.367238 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:15:07.368176 25609 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: attempting to promote NON_VOTER 50e69706d13b44fab124875994d52d40 to VOTER
I20260811 22:15:07.368723 25609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } }
I20260811 22:15:07.370554 25666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 LEARNER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:15:07.370909 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.369943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38170)
0811 22:15:07.370272 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.370807 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:32911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251}
I20260811 22:15:07.371625 25609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:15:07.371971 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.371399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38172)
0811 22:15:07.371691 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.371861 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:32911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":110,"threads_started":1}
W20260811 22:15:07.372182 24969 consensus_peers.cc:487] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 -> Peer b95c531fa328449d8a11730a56385acc (127.17.198.60:32911): Couldn't send request to peer b95c531fa328449d8a11730a56385acc. Status: Network error: Client connection negotiation failed: client connection to 127.17.198.60:32911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:15:07.374533 25665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 50e69706d13b44fab124875994d52d40 (127.17.198.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.374732 25612 master.cc:552] Master@127.17.198.60:32911 shutdown complete.
I20260811 22:15:07.374784 25612 master_runner.cc:305] Clearing existing system tablet
I20260811 22:15:07.375067 25612 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.375128 25612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.376566 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.376711 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.376950 25608 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 50e69706d13b44fab124875994d52d40 (127.17.198.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.377370 25612 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:15:07.378810 25608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.379071 25608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:07.378785 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.379336 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:07.385066 25827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.385411 25612 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.385588 25612 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260811 22:15:07.385638 25612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "b95c531fa328449d8a11730a56385acc"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-77v9"
server_key: "3e5575da025b9a2d1f05cb50c7f50f5d"
server_key_iv: "f351680bdfe088414aa3937dce1eb30e"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.388057 25612 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:07.390847 25612 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:15:07.390980 25612 log.cc:1196] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:15:07.391256 25612 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Deleting consensus metadata
I20260811 22:15:07.391458 25612 master_runner.cc:315] Copying system tablet from 127.17.198.62:42627
I20260811 22:15:07.392747 25612 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: tablet copy: Beginning tablet copy session from remote peer at address 127.17.198.62:42627
I20260811 22:15:07.407436 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.393552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.17.198.62:42627)
0811 22:15:07.393707 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.393712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.397256 (+ 3544us) server_negotiation.cc:407] Connection header received
0811 22:15:07.397386 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.397390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.397457 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.397573 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.398152 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.399069 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.400411 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.400782 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.400929 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.403147 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.403151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.403154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.403187 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.405192 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.405756 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.405760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.405762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.405879 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.406233 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.406237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.406239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.406537 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.406653 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.407098 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.407248 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:15:07.407454 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.393368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:36678)
0811 22:15:07.397100 (+ 3732us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.397120 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.397283 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.397617 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.397620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.397880 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.397999 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.398006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.399266 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.399282 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.400237 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.400250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.400956 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.400961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.401075 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.402413 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.402431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.402947 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.405342 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.405345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.405348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.405630 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.406004 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.406007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.406009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.406117 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.406658 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.406661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.406780 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.407031 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.407228 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3628,"thread_start_us":98,"threads_started":1}
I20260811 22:15:07.408495 25006 tablet_copy_service.cc:140] P 78efa88636f646e29524b2c9cb4bd772: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b95c531fa328449d8a11730a56385acc ({username='oryx', principal='oryx/127.17.198.60@KRBTEST.COM'} at 127.0.0.1:36678)
I20260811 22:15:07.408622 25006 tablet_copy_service.cc:161] P 78efa88636f646e29524b2c9cb4bd772: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b95c531fa328449d8a11730a56385acc at {username='oryx', principal='oryx/127.17.198.60@KRBTEST.COM'} at 127.0.0.1:36678: session id = b95c531fa328449d8a11730a56385acc-00000000000000000000000000000000
I20260811 22:15:07.409972 25006 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:15:07.411427 25612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:15:07.414103 25612 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: tablet copy: Starting download of 0 data blocks...
I20260811 22:15:07.414338 25612 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: tablet copy: Starting download of 1 WAL segments...
I20260811 22:15:07.416157 25612 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:15:07.418473 25006 tablet_copy_service.cc:342] P 78efa88636f646e29524b2c9cb4bd772: Request end of tablet copy session b95c531fa328449d8a11730a56385acc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.198.60@KRBTEST.COM'} at 127.0.0.1:36678
I20260811 22:15:07.418584 25006 tablet_copy_service.cc:433] P 78efa88636f646e29524b2c9cb4bd772: ending tablet copy session b95c531fa328449d8a11730a56385acc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b95c531fa328449d8a11730a56385acc
I20260811 22:15:07.427268 25612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.429206 25840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.429587 25612 server_base.cc:1034] running on GCE node
W20260811 22:15:07.429854 25838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.433265 25837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.437729 25612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.438174 25612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.439363 25612 hybrid_clock.cc:648] HybridClock initialized: now 1786486507439335 us; error 61 us; skew 500 ppm
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.441438 25612 init.cc:377] Logged in from keytab as oryx/127.17.198.60@KRBTEST.COM (short username oryx)
I20260811 22:15:07.442045 25612 webserver.cc:460] Webserver started at http://127.17.198.60:36509/ using document root <none> and password file <none>
I20260811 22:15:07.442282 25612 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.442340 25612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.443938 25612 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:07.445267 25848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.445597 25612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:07.445729 25612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "b95c531fa328449d8a11730a56385acc"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-77v9"
server_key: "3e5575da025b9a2d1f05cb50c7f50f5d"
server_key_iv: "f351680bdfe088414aa3937dce1eb30e"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.445902 25612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.462471 25612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.464988 25612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.470834 25612 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.60:32911
I20260811 22:15:07.473085 25901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.60:32911 every 8 connection(s)
I20260811 22:15:07.473793 25612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
I20260811 22:15:07.475811 25902 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:15:07.477208 25902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Bootstrap starting.
W20260811 22:15:07.483278 25817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:07.483592 25817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:07.483665 25817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:07.488941 25817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:07.489044 25817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:07.489143 25817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:07.489771 25902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:07.490201 25902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: Bootstrap complete.
I20260811 22:15:07.490854 25902 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } }
I20260811 22:15:07.491000 25902 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b95c531fa328449d8a11730a56385acc, State: Initialized, Role: LEARNER
I20260811 22:15:07.491109 25902 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } }
I20260811 22:15:07.491487 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.491595 25902 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:07.491690 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: LEARNER
I20260811 22:15:07.494908 25923 catalog_manager.cc:1259] Loaded cluster ID: f644e6748d694942ab3c59635b1cd0f4
I20260811 22:15:07.495055 25923 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: loading cluster ID for follower catalog manager: success
I20260811 22:15:07.497030 25817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:41559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=46269
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:07.497434 25923 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: acquiring CA information for follower catalog manager: success
I20260811 22:15:07.498390 25817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:07.498797 25923 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:15:07.499547 25817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.510457 25926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.510488 25925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.510463 25928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.510991 25817 server_base.cc:1034] running on GCE node
I20260811 22:15:07.511580 25817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.512382 25817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.513571 25817 hybrid_clock.cc:648] HybridClock initialized: now 1786486507513503 us; error 95 us; skew 500 ppm
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.517278 25817 init.cc:377] Logged in from keytab as oryx/127.17.198.1@KRBTEST.COM (short username oryx)
I20260811 22:15:07.518436 25817 webserver.cc:460] Webserver started at http://127.17.198.1:46269/ using document root <none> and password file <none>
I20260811 22:15:07.519158 25817 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.519259 25817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.524147 25817 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:07.526684 25938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.528204 25817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:07.528379 25817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "01a6303526d44853b8761ef8c98f3699"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "0c7ab3eb1a03ec5193d2912ea6edef2b"
server_key_iv: "8caa6b67bf48421bca19be13eefc55e4"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.529227 25817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.562968 25817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.566856 25817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:07.567140 25817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.567903 25817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:07.568980 25817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:07.569037 25817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.569087 25817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:07.569104 25817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.583575 25817 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:41559
I20260811 22:15:07.583635 26058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:41559 every 8 connection(s)
I20260811 22:15:07.584771 25817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:07.592958 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 25817
I20260811 22:15:07.593199 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25199
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
I20260811 22:15:07.599962 25497 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
I20260811 22:15:07.605619 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:41433
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:42627
--webserver_port=41299
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:07.607266 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.586985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:52663 (local address 127.17.198.61:45713)
0811 22:15:07.587125 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.587130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.589106 (+ 1976us) server_negotiation.cc:407] Connection header received
0811 22:15:07.589792 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.589796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.589860 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.589915 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.591390 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592241 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.597488 (+ 5247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.597880 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.598074 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.601570 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.601577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.601583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.601624 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.603939 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.605096 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.605103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.605108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.605222 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.605691 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.605697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.605699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.605958 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.606055 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.606921 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.607071 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:15:07.607445 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.587062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:51717 (local address 127.17.198.60:32911)
0811 22:15:07.587317 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.587321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.588457 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 22:15:07.589754 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.589757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.589824 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.589915 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.591364 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592277 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.596393 (+ 4116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.596723 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.596834 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.601546 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.601550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.601553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.601588 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.603837 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.604749 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.604756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.604761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.605046 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.605861 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.605867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.605872 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.606157 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.606254 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.606975 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.607154 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260811 22:15:07.609515 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.587499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.17.198.1:52663)
0811 22:15:07.588983 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.588999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.589535 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.590128 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.590136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.590677 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.591204 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.591221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592517 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.592519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.597330 (+ 4811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.597346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.598074 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.598078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.598221 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.598776 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.598788 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.600787 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.604106 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.604110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.604113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.604844 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.605445 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.605449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.605452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.605552 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.606142 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.606153 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.606603 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.607548 (+ 945us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.608082 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1345,"mutex_wait_us":233,"spinlock_wait_cycles":26752,"thread_start_us":334,"threads_started":1}
I20260811 22:15:07.609951 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.587500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.17.198.1:51717)
0811 22:15:07.588278 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.588336 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.589512 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.590128 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.590135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.590603 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.591202 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.591216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592446 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.592450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.596178 (+ 3728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.596200 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.597096 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.597100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.597480 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.598432 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.598462 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.601340 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.604001 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.604012 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.604039 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.604614 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.605216 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.605219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.605220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.605725 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.606321 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.606327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.606603 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.607675 (+ 1072us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.608302 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":313,"spinlock_wait_cycles":106112,"thread_start_us":342,"threads_started":1}
I20260811 22:15:07.611593 26061 heartbeater.cc:347] Connected to a master server at 127.17.198.61:45713
I20260811 22:15:07.611882 26061 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.612583 26061 heartbeater.cc:511] Master 127.17.198.61:45713 requested a full tablet report, sending...
I20260811 22:15:07.612947 26059 heartbeater.cc:347] Connected to a master server at 127.17.198.60:32911
I20260811 22:15:07.613065 26059 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.613334 26059 heartbeater.cc:511] Master 127.17.198.60:32911 requested a full tablet report, sending...
I20260811 22:15:07.613824 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.587737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:55647 (local address 127.17.198.62:42627)
0811 22:15:07.587874 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.587881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.589173 (+ 1292us) server_negotiation.cc:407] Connection header received
0811 22:15:07.589792 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.589796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.589855 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.589977 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.591378 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592250 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.602814 (+ 10564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.603263 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.603371 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.607000 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.607004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.607007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.607041 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.609920 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.611710 (+ 1790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.611715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.611717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.611941 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.612432 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.612436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.612437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.612700 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.612873 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.613476 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.613638 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:15:07.614507 25654 ts_manager.cc:194] Registered new tserver with Master: 01a6303526d44853b8761ef8c98f3699 (127.17.198.1:41559)
I20260811 22:15:07.614707 25858 ts_manager.cc:194] Registered new tserver with Master: 01a6303526d44853b8761ef8c98f3699 (127.17.198.1:41559)
I20260811 22:15:07.614784 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.587607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:55647)
0811 22:15:07.589056 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.589070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.589535 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.590128 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.590136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.590677 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.591207 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.591221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.592470 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.592477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.602632 (+ 10155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.602646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.603683 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.603690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.603848 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.604474 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.604490 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.606698 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.611016 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.611019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.611022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.611521 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.612188 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.612191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.612193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.612304 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.612994 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.612997 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.613247 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.614302 (+ 1055us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.614485 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1379,"mutex_wait_us":206,"spinlock_wait_cycles":25472,"thread_start_us":844,"threads_started":1}
I20260811 22:15:07.615764 26060 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:07.615880 26060 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.616110 26060 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
I20260811 22:15:07.617133 24984 ts_manager.cc:194] Re-registered known tserver with Master: 01a6303526d44853b8761ef8c98f3699 (127.17.198.1:41559)
I20260811 22:15:07.617751 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:55647
W20260811 22:15:07.738282 26069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:07.738605 26069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:07.738651 26069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:07.743412 26069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:07.743500 26069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:07.743562 26069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:07.749701 26069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:41433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=41299
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:07.750931 26069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:07.752110 26069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.764377 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.764377 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.764392 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.764843 26069 server_base.cc:1034] running on GCE node
I20260811 22:15:07.765283 26069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.766016 26069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.767264 26069 hybrid_clock.cc:648] HybridClock initialized: now 1786486507767233 us; error 34 us; skew 500 ppm
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.771564 26069 init.cc:377] Logged in from keytab as oryx/127.17.198.2@KRBTEST.COM (short username oryx)
I20260811 22:15:07.772789 26069 webserver.cc:460] Webserver started at http://127.17.198.2:41299/ using document root <none> and password file <none>
I20260811 22:15:07.773550 26069 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.773689 26069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.778473 26069 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:15:07.781204 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.782763 26069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:07.782956 26069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "79968fcee6cb448d9856748434cd2600"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "5c323387bef7a19449f6b514bba88150"
server_key_iv: "0492b6eb41d1d966cb3544e0204c2035"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.783645 26069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.798537 26069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.802525 26069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:07.802801 26069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.803529 26069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:07.804550 26069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:07.804634 26069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.804718 26069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:07.804790 26069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.816476 26069 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:41433
I20260811 22:15:07.816499 26212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:41433 every 8 connection(s)
I20260811 22:15:07.817808 26069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:07.826748 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26069
I20260811 22:15:07.826970 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25352
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:07.841984 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:42213
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:42627
--webserver_port=37347
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:07.842628 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:56499 (local address 127.17.198.61:45713)
0811 22:15:07.820798 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.820802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.821953 (+ 1151us) server_negotiation.cc:407] Connection header received
0811 22:15:07.822941 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.822944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.822994 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.823105 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.824736 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825616 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.832083 (+ 6467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.832441 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.832533 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.837511 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.837517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.837520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.837558 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.839620 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.840521 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.840525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.840527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.840622 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.841111 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.841117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.841122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.841375 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.841472 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.842313 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.842461 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:15:07.842664 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:58057 (local address 127.17.198.60:32911)
0811 22:15:07.820400 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.820405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.822112 (+ 1707us) server_negotiation.cc:407] Connection header received
0811 22:15:07.823248 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.823251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.823304 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.823394 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.824803 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825472 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.833556 (+ 8084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.834050 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.834162 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.837540 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.837544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.837546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.837577 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.840221 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.840761 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.840765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.840767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.840882 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.841195 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.841198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.841200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.841415 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.841494 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.842400 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.842534 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:15:07.844624 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.17.198.2:56499)
0811 22:15:07.821770 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.821784 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.822458 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.823218 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.823227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.823876 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.824571 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.824586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825783 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.825786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.831860 (+ 6074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.831888 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.832792 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.832887 (+ 95us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.833226 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.834098 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.834117 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.837069 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.839786 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.839794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.839816 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.840390 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.840740 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.840743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.840745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.840998 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.841547 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.841554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.841937 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.842916 (+ 979us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.843599 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1178,"mutex_wait_us":827,"spinlock_wait_cycles":4608,"thread_start_us":124,"threads_started":1}
I20260811 22:15:07.845063 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.17.198.2:58057)
0811 22:15:07.821961 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.821984 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.822473 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.823445 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.823449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.823935 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.824574 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.824591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825667 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.825670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.833352 (+ 7682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.833371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.834384 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.834390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.834529 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.835129 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.835150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.837203 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.840387 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.840390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.840393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.840636 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.840998 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.841002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.841003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.841092 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.841547 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.841554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.841937 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.842963 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.843405 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"mutex_wait_us":296,"spinlock_wait_cycles":50944,"thread_start_us":197,"threads_started":1}
I20260811 22:15:07.846443 26215 heartbeater.cc:347] Connected to a master server at 127.17.198.61:45713
I20260811 22:15:07.846750 26215 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.846823 26213 heartbeater.cc:347] Connected to a master server at 127.17.198.60:32911
I20260811 22:15:07.846935 26213 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.847527 26213 heartbeater.cc:511] Master 127.17.198.60:32911 requested a full tablet report, sending...
I20260811 22:15:07.847527 26215 heartbeater.cc:511] Master 127.17.198.61:45713 requested a full tablet report, sending...
I20260811 22:15:07.848752 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:39223 (local address 127.17.198.62:42627)
0811 22:15:07.820263 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.820268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.821364 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 22:15:07.822799 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.822806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.822888 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.823010 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.824793 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825618 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.837716 (+ 12098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.838226 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.838384 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.842494 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.842502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.842507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.842549 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.845371 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.846497 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.846504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.846509 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.846613 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.847515 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.847522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.847527 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.847825 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.847923 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.848431 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.848595 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:15:07.848953 25858 ts_manager.cc:194] Registered new tserver with Master: 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:07.848989 25654 ts_manager.cc:194] Registered new tserver with Master: 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:07.849545 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.820500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:39223)
0811 22:15:07.821191 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.821246 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.822458 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.823218 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.823240 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.823934 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.824574 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.824591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.825749 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.825754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.837518 (+ 11764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.837536 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.838732 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.838739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.839104 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.839818 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.839864 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.842280 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.846043 (+ 3763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.846049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.846054 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.846339 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.846736 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.846741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.846746 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.847375 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.847991 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.847996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.848118 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.849008 (+ 890us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.849183 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":270,"spinlock_wait_cycles":12416,"thread_start_us":796,"threads_started":1}
I20260811 22:15:07.850261 26214 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:07.850364 26214 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:07.850623 26214 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
I20260811 22:15:07.851629 24984 ts_manager.cc:194] Re-registered known tserver with Master: 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:07.852281 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:39223
I20260811 22:15:07.863941 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.859671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38176)
0811 22:15:07.859871 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.859888 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.860001 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.860271 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.860274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.860291 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:07.860405 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.860412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.861345 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.861349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.862601 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.862614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.863357 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.863363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.863479 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.863523 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.863624 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.863739 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:15:07.863953 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.859808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.17.198.60:32911)
0811 22:15:07.860003 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.860007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.860024 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:07.860084 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.860086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.860144 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.860237 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:07.860572 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.861225 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.862759 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.863212 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.863323 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.863467 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.863672 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.863848 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 22:15:07.864744 25868 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:15:07.865371 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78efa88636f646e29524b2c9cb4bd772. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:07.865569 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: LEARNER
W20260811 22:15:07.974903 26221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:07.975238 26221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:07.975286 26221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:07.979700 26221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:07.979806 26221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:07.979871 26221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:07.985987 26221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:42213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=37347
--tserver_master_addrs=127.17.198.62:42627,127.17.198.61:45713,127.17.198.60:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:07.987308 26221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:07.988478 26221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.999614 26230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.999634 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.999634 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:08.000043 26221 server_base.cc:1034] running on GCE node
I20260811 22:15:08.000643 26221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:08.001433 26221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:08.002641 26221 hybrid_clock.cc:648] HybridClock initialized: now 1786486508002604 us; error 50 us; skew 500 ppm
Aug 11 22:15:08 dist-test-slave-77v9 krb5kdc[24930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486508, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:08.006359 26221 init.cc:377] Logged in from keytab as oryx/127.17.198.3@KRBTEST.COM (short username oryx)
I20260811 22:15:08.007715 26221 webserver.cc:460] Webserver started at http://127.17.198.3:37347/ using document root <none> and password file <none>
I20260811 22:15:08.008379 26221 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:08.008481 26221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:08.013396 26221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:08.016036 26237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:08.017645 26221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:08.017813 26221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-77v9"
server_key: "285f34404db921cf714eeef182f7035c"
server_key_iv: "d64b14eda8077db9d66689c7de4d0733"
server_key_version: "encryptionkey@0"
I20260811 22:15:08.018538 26221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:08.052676 26221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:08.057073 26221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:08.057334 26221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:08.058079 26221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:08.059116 26221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:08.059198 26221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:08.059283 26221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:08.059330 26221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:08.071805 26221 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:42213
I20260811 22:15:08.071871 26365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:42213 every 8 connection(s)
I20260811 22:15:08.073467 26221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:08.083514 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26221
Aug 11 22:15:08 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486508, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.61@KRBTEST.COM
Aug 11 22:15:08 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486508, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.60@KRBTEST.COM
Aug 11 22:15:08 dist-test-slave-77v9 krb5kdc[24930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486508, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:08.097443 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.076500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.17.198.3:37265)
0811 22:15:08.077744 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.077764 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.078728 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.079402 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.079411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.079969 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:08.080722 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.080733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.082064 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.082068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.083382 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.083396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.084378 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.084869 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.086228 (+ 1359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:08.086253 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:08.090380 (+ 4127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:08.092971 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.092979 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.093002 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.093473 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.093820 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.093823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.093825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.094068 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.094655 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:08.094664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:08.095112 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.095371 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.095812 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"mutex_wait_us":389,"spinlock_wait_cycles":1250944,"thread_start_us":129,"threads_started":1}
I20260811 22:15:08.098472 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.076086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:37265 (local address 127.17.198.60:32911)
0811 22:15:08.076215 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.076220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.077864 (+ 1644us) server_negotiation.cc:407] Connection header received
0811 22:15:08.078981 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.078987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.079057 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.079159 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.080941 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.081811 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.083598 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.083986 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084082 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.090588 (+ 6506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.090595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.090601 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.090636 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.092829 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.093582 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.093585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.093587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.093693 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.094207 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.094211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.094212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.094457 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.094590 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.098122 (+ 3532us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.098282 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:15:08.099772 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.077040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:36483 (local address 127.17.198.61:45713)
0811 22:15:08.077182 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.077187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.078073 (+ 886us) server_negotiation.cc:407] Connection header received
0811 22:15:08.079104 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.079107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.079196 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.079286 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.080988 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.081805 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.083670 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.084071 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084150 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.091365 (+ 7215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.091371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.091378 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.091417 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.093450 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.094378 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.094385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.094390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.094500 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.096959 (+ 2459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.096964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.096966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.097308 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.097446 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.099431 (+ 1985us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.099582 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:15:08.099766 26366 heartbeater.cc:347] Connected to a master server at 127.17.198.60:32911
I20260811 22:15:08.100083 26366 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:08.100795 26366 heartbeater.cc:511] Master 127.17.198.60:32911 requested a full tablet report, sending...
I20260811 22:15:08.101117 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.076851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.17.198.3:36483)
0811 22:15:08.077980 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.077992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.078728 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.079418 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.079420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.079993 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:08.080779 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.080798 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.082064 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.082068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.083515 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.083528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.084482 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.084972 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.086054 (+ 1082us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:08.086076 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:08.091103 (+ 5027us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x26ac86b8. stack: 00007f4790852c78 00007f47908531a9 00007f47904151d7 00000000005984a6 00000000005984f9 000000000059c488 00007f479084f2ab 00007f479084ead0 00007f4794f270ee 00007f4794ea9cae 00007f4794ea907b 00007f4794ea5e17 00007f4794ee6832 00007f4794ee768c 00007f4794efd0d3 00007f4794f0054b
0811 22:15:08.091155 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:08.093981 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.093985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.093987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.094241 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.095005 (+ 764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.095008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.095010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.095111 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.099051 (+ 3940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:08.099058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:08.099186 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.099937 (+ 751us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.100127 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1046,"mutex_wait_us":287,"spinlock_wait_cycles":2405248,"thread_start_us":101,"threads_started":1}
I20260811 22:15:08.102173 26368 heartbeater.cc:347] Connected to a master server at 127.17.198.61:45713
I20260811 22:15:08.102294 26368 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:08.102547 26368 heartbeater.cc:511] Master 127.17.198.61:45713 requested a full tablet report, sending...
I20260811 22:15:08.102711 25858 ts_manager.cc:194] Registered new tserver with Master: b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213)
I20260811 22:15:08.103609 25654 ts_manager.cc:194] Registered new tserver with Master: b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213)
I20260811 22:15:08.104054 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.087775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.17.198.60:32911)
0811 22:15:08.089751 (+ 1976us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.089768 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.089784 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:08.089834 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.089836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.089929 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.090054 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.093201 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.094326 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.095523 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.095971 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.096090 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.098652 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.098656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.098659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.098689 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.100963 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.101824 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.101831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.101834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.101947 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.102597 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.102601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.102602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.102970 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.103100 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.103788 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.103927 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1882,"thread_start_us":139,"threads_started":1}
I20260811 22:15:08.105981 26372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.076500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:34527)
0811 22:15:08.077511 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.077584 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.078693 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.079400 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.079409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.079965 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:08.080709 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.080727 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.081783 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.081787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.083358 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.083372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.084391 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.084862 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.091644 (+ 6782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:08.091657 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:08.093941 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:08.097997 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.098003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.098008 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.098296 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.102802 (+ 4506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:08.102807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:08.102813 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:08.103020 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:08.104549 (+ 1529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:08.104552 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:08.104684 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.105536 (+ 852us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.105748 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":200,"spinlock_wait_cycles":5888,"thread_start_us":273,"threads_started":1}
I20260811 22:15:08.106045 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.075920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:34527 (local address 127.17.198.62:42627)
0811 22:15:08.076147 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.076153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.077769 (+ 1616us) server_negotiation.cc:407] Connection header received
0811 22:15:08.079025 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.079028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.079091 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.079158 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.080956 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.081604 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.083580 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.084008 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.084109 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.094802 (+ 10693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.094805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.094808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.094830 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.097464 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.102178 (+ 4714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.102446 (+ 268us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.102452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.102652 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.103159 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.103164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.103167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.103596 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.103725 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.105671 (+ 1946us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.105864 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:15:08.107201 26367 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42627
I20260811 22:15:08.107384 26367 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:08.107801 26367 heartbeater.cc:511] Master 127.17.198.62:42627 requested a full tablet report, sending...
I20260811 22:15:08.108788 24984 ts_manager.cc:194] Re-registered known tserver with Master: b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213)
I20260811 22:15:08.109731 24984 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:34527
I20260811 22:15:08.111610 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.088447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.17.198.61:45713)
0811 22:15:08.089751 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.089757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.090638 (+ 881us) server_negotiation.cc:407] Connection header received
0811 22:15:08.096332 (+ 5694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.096337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.096541 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.096693 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.098009 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.098976 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.100284 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.100720 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.100992 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.105902 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.105907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.105911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.105945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.108848 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.109538 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.109542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.109543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.109635 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.110381 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.110385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.110387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.110673 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.110821 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.111244 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.111417 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1209,"thread_start_us":103,"threads_started":1}
I20260811 22:15:08.113631 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.086661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.17.198.62:42627)
0811 22:15:08.087298 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.087334 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.087507 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:15:08.087802 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.087806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.087870 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.088003 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.089637 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.090616 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.094109 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.094486 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.094612 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.098463 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.098467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.098470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.098501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.108256 (+ 9755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.110128 (+ 1872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.110133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.110135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.110244 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.112435 (+ 2191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.112440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.112442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.112696 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.112946 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.113217 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.113397 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":114,"threads_started":1}
I20260811 22:15:08.117404 24984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:08.121233 24984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:08.124627 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: NON_VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: true } } }
I20260811 22:15:08.124902 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:08.128885 26388 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: attempting to promote NON_VOTER b95c531fa328449d8a11730a56385acc to VOTER
I20260811 22:15:08.129336 26388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:08.130632 25666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:15:08.130707 25868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 LEARNER]: Refusing update from remote peer 78efa88636f646e29524b2c9cb4bd772: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:15:08.131321 25828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:15:08.131527 26388 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:15:08.133927 25828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b95c531fa328449d8a11730a56385acc (127.17.198.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.136052 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.136201 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:08.136302 26389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.136463 26389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:08.137084 25665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b95c531fa328449d8a11730a56385acc (127.17.198.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.137997 26389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.138139 26389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:08.138552 25868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b95c531fa328449d8a11730a56385acc (127.17.198.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.139442 26399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.139631 26399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:08.141820 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78efa88636f646e29524b2c9cb4bd772" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:08.141964 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:08.150100 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.144131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41559 (local address 127.0.0.1:53832)
0811 22:15:08.144310 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.144330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.144423 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.145518 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.145522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.145541 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:08.145654 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.145661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.147640 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.147644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.148619 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.148629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.149683 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.149687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.149778 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.149810 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.149865 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.149940 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":36}
I20260811 22:15:08.151087 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.144601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.17.198.1:41559)
0811 22:15:08.144914 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.144945 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.144974 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:15:08.145062 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.145084 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.145377 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.145577 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:08.146073 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.147490 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.148757 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.149534 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.150448 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.150769 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.150827 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.150943 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":112,"threads_started":1}
I20260811 22:15:08.151082 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.144839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:41433 (local address 127.0.0.1:55330)
0811 22:15:08.145162 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.145177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.145264 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.146625 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.146629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.146651 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:08.146758 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.146764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.148117 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.148122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.149475 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.149486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.150632 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.150636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.150731 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.150756 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.150827 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.150926 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":156,"threads_started":1}
I20260811 22:15:08.151113 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.145493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.17.198.2:41433)
0811 22:15:08.146093 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.146105 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.146129 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:15:08.146208 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.146213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.146434 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.146683 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:08.146875 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.147993 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.149613 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.150480 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.150631 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.150789 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.150868 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.150965 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":332,"threads_started":1}
I20260811 22:15:08.153235 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.144353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:42213 (local address 127.0.0.1:53388)
0811 22:15:08.144727 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.144743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.144865 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.147506 (+ 2641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.147510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.147535 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:08.147717 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.147724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.149350 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.149353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.150519 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.150603 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.152626 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.152630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.152729 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.152759 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.152997 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.153084 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":51,"thread_start_us":221,"threads_started":1}
I20260811 22:15:08.153319 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.145052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.17.198.3:42213)
0811 22:15:08.146997 (+ 1945us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.147021 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.147046 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:15:08.147125 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.147130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.147306 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.147501 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:08.147863 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.149201 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.151398 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.152450 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.152570 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.152717 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.153053 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.153163 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1273,"thread_start_us":109,"threads_started":1}
I20260811 22:15:08.154649 26140 tablet_service.cc:1467] Processing CreateTablet for tablet 75fd96c303fe4561a85582078f200caa (DEFAULT_TABLE table=test-table [id=918a2bbee10a4630ad7be84eb6125e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:08.154675 25990 tablet_service.cc:1467] Processing CreateTablet for tablet 75fd96c303fe4561a85582078f200caa (DEFAULT_TABLE table=test-table [id=918a2bbee10a4630ad7be84eb6125e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:08.155969 26140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75fd96c303fe4561a85582078f200caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:08.155982 25990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75fd96c303fe4561a85582078f200caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:08.161619 26290 tablet_service.cc:1467] Processing CreateTablet for tablet 75fd96c303fe4561a85582078f200caa (DEFAULT_TABLE table=test-table [id=918a2bbee10a4630ad7be84eb6125e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:08.163372 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75fd96c303fe4561a85582078f200caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:08.166597 26405 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Bootstrap starting.
I20260811 22:15:08.169138 26405 tablet_bootstrap.cc:654] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:08.170235 26405 log.cc:824] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:08.171428 26406 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Bootstrap starting.
I20260811 22:15:08.173609 26405 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: No bootstrap required, opened a new log
I20260811 22:15:08.173844 26405 ts_tablet_manager.cc:1397] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:15:08.174405 26406 tablet_bootstrap.cc:654] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:08.175227 26408 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Bootstrap starting.
I20260811 22:15:08.175701 26406 log.cc:824] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:08.178593 26408 tablet_bootstrap.cc:654] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:08.178639 26406 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: No bootstrap required, opened a new log
I20260811 22:15:08.178881 26406 ts_tablet_manager.cc:1397] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 22:15:08.178948 26405 raft_consensus.cc:348] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.179284 26405 raft_consensus.cc:374] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:08.179342 26405 raft_consensus.cc:729] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a6303526d44853b8761ef8c98f3699, State: Initialized, Role: FOLLOWER
I20260811 22:15:08.179816 26405 consensus_queue.cc:260] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.180035 26408 log.cc:824] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:08.181058 26405 ts_tablet_manager.cc:1428] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:15:08.182893 26408 tablet_bootstrap.cc:492] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: No bootstrap required, opened a new log
I20260811 22:15:08.183166 26408 ts_tablet_manager.cc:1397] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260811 22:15:08.183583 26060 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
I20260811 22:15:08.188294 26406 raft_consensus.cc:348] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.188669 26406 raft_consensus.cc:374] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:08.188745 26406 raft_consensus.cc:729] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79968fcee6cb448d9856748434cd2600, State: Initialized, Role: FOLLOWER
I20260811 22:15:08.189446 26406 consensus_queue.cc:260] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.190908 26406 ts_tablet_manager.cc:1428] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 22:15:08.191283 26214 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
I20260811 22:15:08.194715 26408 raft_consensus.cc:348] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.195178 26408 raft_consensus.cc:374] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:08.195329 26408 raft_consensus.cc:729] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c7e0ee92954ec2ab62caaa0442aff8, State: Initialized, Role: FOLLOWER
I20260811 22:15:08.195892 26408 consensus_queue.cc:260] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.196913 26367 heartbeater.cc:503] Master 127.17.198.62:42627 was elected leader, sending a full tablet report...
I20260811 22:15:08.197206 26408 ts_tablet_manager.cc:1428] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Time spent starting tablet: real 0.014s user 0.001s sys 0.009s
I20260811 22:15:08.205036 26410 raft_consensus.cc:482] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:08.205281 26410 raft_consensus.cc:504] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.206452 26410 leader_election.cc:288] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213), 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:08.211772 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.206782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:42213 (local address 127.17.198.1:38131)
0811 22:15:08.207527 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.207556 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.207967 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.208308 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.208315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.208340 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:08.208626 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.208634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.209547 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.209551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.210538 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.210548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.211223 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.211227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.211376 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.211398 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.211492 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.211574 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":341,"threads_started":1}
I20260811 22:15:08.211728 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.206862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38131 (local address 127.17.198.3:42213)
0811 22:15:08.207038 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.207043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.207705 (+ 662us) server_negotiation.cc:407] Connection header received
0811 22:15:08.208133 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.208135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.208181 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.208252 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:08.208870 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.209421 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.210675 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.211086 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.211225 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.211283 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.211524 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.211604 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:15:08.212625 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75fd96c303fe4561a85582078f200caa" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" is_pre_election: true
I20260811 22:15:08.212845 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.207916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:57195 (local address 127.17.198.2:41433)
0811 22:15:08.208090 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.208095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.208390 (+ 295us) server_negotiation.cc:407] Connection header received
0811 22:15:08.208500 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.208502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.208541 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.208593 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:08.209341 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.209990 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.211322 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.211912 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.212063 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.212111 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.212597 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.212704 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:15:08.212859 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.207504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:41433 (local address 127.17.198.1:57195)
0811 22:15:08.208259 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:08.208275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:08.208408 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:08.208632 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:08.208635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:08.208654 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:08.209101 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.209108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.210126 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.210133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.211151 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:08.211162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.212177 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.212181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:08.212401 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.212423 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:15:08.212540 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:15:08.212625 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":385,"threads_started":1}
I20260811 22:15:08.213066 26315 raft_consensus.cc:2454] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 0.
I20260811 22:15:08.213759 26166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75fd96c303fe4561a85582078f200caa" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79968fcee6cb448d9856748434cd2600" is_pre_election: true
I20260811 22:15:08.213809 25940 leader_election.cc:302] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a6303526d44853b8761ef8c98f3699, b7c7e0ee92954ec2ab62caaa0442aff8; no voters:
I20260811 22:15:08.214145 26166 raft_consensus.cc:2454] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 0.
I20260811 22:15:08.214365 26410 raft_consensus.cc:2781] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:08.214568 26410 raft_consensus.cc:482] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:08.214661 26410 raft_consensus.cc:3037] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:08.216495 26410 raft_consensus.cc:504] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.217097 26410 leader_election.cc:288] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 election: Requested vote from peers b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213), 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:08.217574 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75fd96c303fe4561a85582078f200caa" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
I20260811 22:15:08.217612 26166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75fd96c303fe4561a85582078f200caa" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79968fcee6cb448d9856748434cd2600"
I20260811 22:15:08.217758 26166 raft_consensus.cc:3037] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:08.217746 26315 raft_consensus.cc:3037] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:08.219204 26166 raft_consensus.cc:2454] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 1.
I20260811 22:15:08.219512 26315 raft_consensus.cc:2454] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 1.
I20260811 22:15:08.219535 25940 leader_election.cc:302] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a6303526d44853b8761ef8c98f3699, 79968fcee6cb448d9856748434cd2600; no voters:
I20260811 22:15:08.219786 26410 raft_consensus.cc:2781] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:08.220163 26410 raft_consensus.cc:686] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 1 LEADER]: Becoming Leader. State: Replica: 01a6303526d44853b8761ef8c98f3699, State: Running, Role: LEADER
I20260811 22:15:08.220621 26410 consensus_queue.cc:237] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } }
I20260811 22:15:08.224351 24983 catalog_manager.cc:5495] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a6303526d44853b8761ef8c98f3699 (127.17.198.1). New cstate: current_term: 1 leader_uuid: "01a6303526d44853b8761ef8c98f3699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } health_report { overall_health: UNKNOWN } } }
I20260811 22:15:08.240510 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.236272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.17.198.1:41559)
0811 22:15:08.236435 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.236440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.236457 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:08.236589 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.236591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.236650 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.236770 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:08.237129 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.237685 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.238753 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.239050 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.239190 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.239556 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:08.240045 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:08.240191 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.240276 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.240348 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:15:08.247913 26166 raft_consensus.cc:1261] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Refusing update from remote peer 01a6303526d44853b8761ef8c98f3699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:08.247965 26315 raft_consensus.cc:1261] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Refusing update from remote peer 01a6303526d44853b8761ef8c98f3699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:08.248828 26418 consensus_queue.cc:1035] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:08.249037 26410 consensus_queue.cc:1035] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:08.261282 26425 mvcc.cc:205] Tried to move back new op lower bound from 7317448737774706688 to 7317448737674874880. Current Snapshot: MvccSnapshot[applied={T|T < 7317448737774706688}]
I20260811 22:15:08.272647 24984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:08.272923 24984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:08.278270 24984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78efa88636f646e29524b2c9cb4bd772: Sending DeleteTablet for 3 replicas of tablet 75fd96c303fe4561a85582078f200caa
I20260811 22:15:08.279663 26140 tablet_service.cc:1514] Processing DeleteTablet for tablet 75fd96c303fe4561a85582078f200caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:08 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:55330
I20260811 22:15:08.279595 26290 tablet_service.cc:1514] Processing DeleteTablet for tablet 75fd96c303fe4561a85582078f200caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:08 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53388
I20260811 22:15:08.280782 25990 tablet_service.cc:1514] Processing DeleteTablet for tablet 75fd96c303fe4561a85582078f200caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:08 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53832
I20260811 22:15:08.281232 26435 tablet_replica.cc:330] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: stopping tablet replica
I20260811 22:15:08.281701 26435 raft_consensus.cc:2229] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:08.282104 26435 raft_consensus.cc:2258] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:08.282557 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 24946
I20260811 22:15:08.287074 26435 ts_tablet_manager.cc:1905] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:08.290777 26435 ts_tablet_manager.cc:1918] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:08.290920 26435 log.cc:1196] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/75fd96c303fe4561a85582078f200caa
I20260811 22:15:08.291493 26436 tablet_replica.cc:330] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: stopping tablet replica
I20260811 22:15:08.291908 26436 raft_consensus.cc:2229] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:08.292330 26438 tablet_replica.cc:330] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: stopping tablet replica
I20260811 22:15:08.292351 26436 raft_consensus.cc:2258] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:08.292698 26438 raft_consensus.cc:2229] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:08.293526 26438 raft_consensus.cc:2258] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:08.294654 26436 ts_tablet_manager.cc:1905] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:08.296340 26435 ts_tablet_manager.cc:1939] T 75fd96c303fe4561a85582078f200caa P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting consensus metadata
W20260811 22:15:08.298044 26240 connection.cc:441] server connection from 127.0.0.1:53388 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53388 (request call id 1) could send its response
I20260811 22:15:08.298117 26438 ts_tablet_manager.cc:1905] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:08.300302 26436 ts_tablet_manager.cc:1918] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:08.300459 26436 log.cc:1196] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/75fd96c303fe4561a85582078f200caa
I20260811 22:15:08.301083 26436 ts_tablet_manager.cc:1939] T 75fd96c303fe4561a85582078f200caa P 79968fcee6cb448d9856748434cd2600: Deleting consensus metadata
I20260811 22:15:08.301508 26438 ts_tablet_manager.cc:1918] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:08.301637 26438 log.cc:1196] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/75fd96c303fe4561a85582078f200caa
I20260811 22:15:08.302028 26438 ts_tablet_manager.cc:1939] T 75fd96c303fe4561a85582078f200caa P 01a6303526d44853b8761ef8c98f3699: Deleting consensus metadata
W20260811 22:15:08.303023 25941 connection.cc:441] server connection from 127.0.0.1:53832 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53832 (request call id 1) could send its response
W20260811 22:15:08.303040 26088 connection.cc:441] server connection from 127.0.0.1:55330 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55330 (request call id 1) could send its response
I20260811 22:15:08.307588 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.280764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.17.198.61:45713)
0811 22:15:08.280907 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.280923 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.282060 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 22:15:08.289257 (+ 7197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.289261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.289335 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.289396 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.290845 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.291701 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.293957 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.294319 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.294422 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.302464 (+ 8042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.302469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.302473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.302509 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.305497 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.306049 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.306053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.306055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.306151 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.306462 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.306465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.306467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.306713 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.306837 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.307273 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.307417 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:15:08.314332 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.303832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.17.198.61:45713)
0811 22:15:08.303973 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.303977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.303993 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:08.304165 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.304169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.304229 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.304287 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.305797 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.306648 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.307978 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.308334 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.308414 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.310300 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.310305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.310307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.310335 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.312338 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.312903 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.312909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.312912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.313004 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.313320 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.313324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.313326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.313543 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.313647 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.314022 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.314158 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:15:08.316604 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.303618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.17.198.60:32911)
0811 22:15:08.303783 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.303788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.304683 (+ 895us) server_negotiation.cc:407] Connection header received
0811 22:15:08.304929 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.304935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.304998 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.305056 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.306002 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.306808 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.309454 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.309807 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.309878 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.311656 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.311659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.311662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.311691 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.313451 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.314062 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.314067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.314069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.314163 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.314989 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.314995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.314998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.315252 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.315383 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.316249 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.316411 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:15:08.318461 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:08.281646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.17.198.60:32911)
0811 22:15:08.281785 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:08.281794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:08.308186 (+ 26392us) server_negotiation.cc:407] Connection header received
0811 22:15:08.308379 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:08.308383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:08.308441 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:08.308494 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:08.309793 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.311151 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.312358 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:08.312778 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:08.312895 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:08.314629 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:08.314633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:08.314635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:08.314659 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:08.316452 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.317103 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.317107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.317109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.317197 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:08.317517 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:08.317520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:08.317521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:08.317719 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:08.317860 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:08.318136 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:15:08.318280 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:15:09.264744 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.264245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:50567)
0811 22:15:09.264548 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.264643 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":126,"threads_started":1}
W20260811 22:15:09.265290 26367 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:42627 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.268528 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.267991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:34887)
0811 22:15:09.268287 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.268378 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
W20260811 22:15:09.269007 26060 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:42627 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.274924 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.274328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:34563)
0811 22:15:09.274695 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.274799 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":121,"threads_started":1}
W20260811 22:15:09.275202 26214 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:42627 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.779325 26453 raft_consensus.cc:482] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78efa88636f646e29524b2c9cb4bd772)
I20260811 22:15:09.779327 26452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78efa88636f646e29524b2c9cb4bd772)
I20260811 22:15:09.779584 26452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:09.779583 26453 raft_consensus.cc:504] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:09.780644 26452 leader_election.cc:288] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), b95c531fa328449d8a11730a56385acc (127.17.198.60:32911)
I20260811 22:15:09.781087 26453 leader_election.cc:288] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), 50e69706d13b44fab124875994d52d40 (127.17.198.61:45713)
I20260811 22:15:09.781471 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.780797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44774)
0811 22:15:09.781122 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.781352 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":76,"threads_started":1}
W20260811 22:15:09.781764 25850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.781864 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.781470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44776)
0811 22:15:09.781681 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.781756 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":56,"threads_started":1}
W20260811 22:15:09.782027 25635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.782976 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.782730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44792)
0811 22:15:09.782858 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.782903 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 22:15:09.783106 25635 leader_election.cc:334] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.784288 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.784027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44800)
0811 22:15:09.784172 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.784223 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 22:15:09.784456 25850 leader_election.cc:334] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.786022 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.781445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.17.198.60:32911)
0811 22:15:09.781956 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:09.781961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:09.781974 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:15:09.782015 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:09.782018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:09.782062 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:09.782144 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:09.782543 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.783287 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785024 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.785426 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785555 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:09.785617 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:09.785784 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:15:09.785878 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":434,"thread_start_us":54,"threads_started":1}
I20260811 22:15:09.786056 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.780886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.17.198.61:45713)
0811 22:15:09.781514 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:09.781521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:09.781537 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:09.781589 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:09.781592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:09.781645 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:09.782043 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:09.782155 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.782883 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785034 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.785440 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785572 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:09.785633 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:09.785805 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:15:09.785893 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":109,"threads_started":1}
I20260811 22:15:09.786067 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.781247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:32911 (local address 127.0.0.1:38032)
0811 22:15:09.781655 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.781670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:09.781764 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:09.782282 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:09.782285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:09.782301 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:09.782396 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:09.782404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.783423 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.783426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:09.784886 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:09.784897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.785556 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:09.785647 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:09.785669 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:15:09.785769 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:15:09.785864 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":70,"threads_started":1}
I20260811 22:15:09.786053 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.780842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:45713 (local address 127.0.0.1:39772)
0811 22:15:09.781121 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.781138 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:09.781258 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:09.781778 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:09.781784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:09.781805 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:09.781921 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:09.781928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.783018 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.783021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:09.784907 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:09.784917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:09.785572 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:09.785576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:09.785667 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:09.785688 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:15:09.785790 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:15:09.785929 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":47,"thread_start_us":81,"threads_started":1}
I20260811 22:15:09.786664 25868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50e69706d13b44fab124875994d52d40" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b95c531fa328449d8a11730a56385acc" is_pre_election: true
I20260811 22:15:09.786769 25665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b95c531fa328449d8a11730a56385acc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50e69706d13b44fab124875994d52d40" is_pre_election: true
I20260811 22:15:09.786924 25665 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b95c531fa328449d8a11730a56385acc in term 1.
I20260811 22:15:09.786906 25868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50e69706d13b44fab124875994d52d40 in term 1.
I20260811 22:15:09.787278 25637 leader_election.cc:302] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50e69706d13b44fab124875994d52d40, b95c531fa328449d8a11730a56385acc; no voters: 78efa88636f646e29524b2c9cb4bd772
I20260811 22:15:09.787473 26452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:15:09.787551 26452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 78efa88636f646e29524b2c9cb4bd772)
I20260811 22:15:09.787607 26452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:15:09.787695 25853 leader_election.cc:302] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50e69706d13b44fab124875994d52d40, b95c531fa328449d8a11730a56385acc; no voters: 78efa88636f646e29524b2c9cb4bd772
I20260811 22:15:09.787936 26453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:15:09.788009 26453 raft_consensus.cc:482] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 78efa88636f646e29524b2c9cb4bd772)
I20260811 22:15:09.788050 26453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:15:09.789136 26452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:09.789399 26453 raft_consensus.cc:504] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:09.789623 26452 leader_election.cc:288] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 election: Requested vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), b95c531fa328449d8a11730a56385acc (127.17.198.60:32911)
I20260811 22:15:09.789809 26453 leader_election.cc:288] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 election: Requested vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), 50e69706d13b44fab124875994d52d40 (127.17.198.61:45713)
I20260811 22:15:09.790048 25868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50e69706d13b44fab124875994d52d40" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b95c531fa328449d8a11730a56385acc"
I20260811 22:15:09.790169 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.789926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44818)
0811 22:15:09.790023 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.790079 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 22:15:09.790233 25868 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50e69706d13b44fab124875994d52d40 in current term 2: Already voted for candidate b95c531fa328449d8a11730a56385acc in this term.
I20260811 22:15:09.790331 25665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b95c531fa328449d8a11730a56385acc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50e69706d13b44fab124875994d52d40"
I20260811 22:15:09.790500 25665 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b95c531fa328449d8a11730a56385acc in current term 2: Already voted for candidate 50e69706d13b44fab124875994d52d40 in this term.
I20260811 22:15:09.790804 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.790599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44832)
0811 22:15:09.790688 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.790737 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 22:15:09.790913 25850 leader_election.cc:334] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.790951 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.789789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44812)
0811 22:15:09.790193 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.790254 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325}
I20260811 22:15:09.790992 25850 leader_election.cc:302] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b95c531fa328449d8a11730a56385acc; no voters: 50e69706d13b44fab124875994d52d40, 78efa88636f646e29524b2c9cb4bd772
I20260811 22:15:09.791301 26453 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:15:09.791682 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:09.791419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:44836)
0811 22:15:09.791585 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:09.791632 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 22:15:09.791792 25635 leader_election.cc:334] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:09.791910 25635 leader_election.cc:302] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50e69706d13b44fab124875994d52d40; no voters: 78efa88636f646e29524b2c9cb4bd772, b95c531fa328449d8a11730a56385acc
I20260811 22:15:09.792109 26452 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:15:10.266611 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:10.266066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:36603)
0811 22:15:10.266342 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:10.266410 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260811 22:15:10.270095 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:10.269506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:32985)
0811 22:15:10.269808 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:10.269871 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260811 22:15:10.276556 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:10.275888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:47973)
0811 22:15:10.276336 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:10.276436 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260811 22:15:11.268303 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.267699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:48819)
0811 22:15:11.268103 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.268185 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":189,"threads_started":1}
I20260811 22:15:11.271335 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.270804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:55663)
0811 22:15:11.271154 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.271238 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260811 22:15:11.278294 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.277688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:60087)
0811 22:15:11.278029 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.278121 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260811 22:15:11.573297 26471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:11.573472 26471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:11.574153 26471 leader_election.cc:288] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), b95c531fa328449d8a11730a56385acc (127.17.198.60:32911)
I20260811 22:15:11.574878 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.574377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45116)
0811 22:15:11.574663 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.574759 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260811 22:15:11.574954 25868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50e69706d13b44fab124875994d52d40" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b95c531fa328449d8a11730a56385acc" is_pre_election: true
I20260811 22:15:11.575264 25868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50e69706d13b44fab124875994d52d40 in term 2.
I20260811 22:15:11.575769 25637 leader_election.cc:302] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50e69706d13b44fab124875994d52d40, b95c531fa328449d8a11730a56385acc; no voters:
I20260811 22:15:11.576071 26471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 22:15:11.576149 26471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:11.576184 26471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:15:11.576346 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.575862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45130)
0811 22:15:11.576134 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.576229 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260811 22:15:11.576488 25635 leader_election.cc:334] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:11.578281 26471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:11.578888 26471 leader_election.cc:288] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 election: Requested vote from peers 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627), b95c531fa328449d8a11730a56385acc (127.17.198.60:32911)
I20260811 22:15:11.579492 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.579126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45138)
0811 22:15:11.579318 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.579380 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:15:11.579463 25868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50e69706d13b44fab124875994d52d40" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b95c531fa328449d8a11730a56385acc"
I20260811 22:15:11.579650 25868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:15:11.580561 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.580255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45146)
0811 22:15:11.580394 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.580469 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:15:11.580699 25635 leader_election.cc:334] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
I20260811 22:15:11.581359 25868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50e69706d13b44fab124875994d52d40 in term 3.
I20260811 22:15:11.581905 25637 leader_election.cc:302] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50e69706d13b44fab124875994d52d40, b95c531fa328449d8a11730a56385acc; no voters: 78efa88636f646e29524b2c9cb4bd772
I20260811 22:15:11.582275 26471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:15:11.582547 26471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [term 3 LEADER]: Becoming Leader. State: Replica: 50e69706d13b44fab124875994d52d40, State: Running, Role: LEADER
I20260811 22:15:11.582850 26471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } }
I20260811 22:15:11.584306 26474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e69706d13b44fab124875994d52d40. Latest consensus state: current_term: 3 leader_uuid: "50e69706d13b44fab124875994d52d40" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:11.584559 26474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:11.585007 26476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:11.587993 26476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:11.588603 26476 catalog_manager.cc:1259] Loaded cluster ID: f644e6748d694942ab3c59635b1cd0f4
I20260811 22:15:11.588676 26476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:11.589392 26476 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:11.589825 26476 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Loaded TSK: 0
I20260811 22:15:11.590626 26476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:11.632668 25654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:11.636756 25654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:11.640744 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.640402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45162)
0811 22:15:11.640565 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.640641 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:15:11.640854 25868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [term 3 FOLLOWER]: Refusing update from remote peer 50e69706d13b44fab124875994d52d40: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 22:15:11.641589 26474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 22:15:11.641785 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.641482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.0.0.1:45168)
0811 22:15:11.641629 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.641704 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:15:11.641991 25635 consensus_peers.cc:487] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 -> Peer 78efa88636f646e29524b2c9cb4bd772 (127.17.198.62:42627): Couldn't send request to peer 78efa88636f646e29524b2c9cb4bd772. Status: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:15:11.643797 26478 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e69706d13b44fab124875994d52d40. Latest consensus state: current_term: 3 leader_uuid: "50e69706d13b44fab124875994d52d40" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:11.644088 26478 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:11.645756 26474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "50e69706d13b44fab124875994d52d40" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:11.645947 26474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:11.647190 26478 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "50e69706d13b44fab124875994d52d40" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "78efa88636f646e29524b2c9cb4bd772" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42627 } } peers { permanent_uuid: "50e69706d13b44fab124875994d52d40" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 45713 } attrs { promote: false } } peers { permanent_uuid: "b95c531fa328449d8a11730a56385acc" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 32911 } attrs { promote: false } } }
I20260811 22:15:11.647367 26478 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c531fa328449d8a11730a56385acc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:11.659861 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.654704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.17.198.3:42213)
0811 22:15:11.655604 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:11.655610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:11.655624 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:15:11.655680 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:11.655685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:11.655742 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:11.655845 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:11.656197 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657236 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.658585 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.659068 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.659274 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.659354 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:11.659608 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 22:15:11.659699 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":128,"threads_started":1}
I20260811 22:15:11.659845 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.654569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:42213 (local address 127.0.0.1:49266)
0811 22:15:11.655188 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.655208 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:11.655317 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:11.655881 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:11.655887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:11.655911 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:11.656043 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.656050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657383 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.657387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.658416 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.658428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.659305 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.659309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.659433 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.659463 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:15:11.659563 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:15:11.659660 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":92}
I20260811 22:15:11.660841 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.654835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:41433 (local address 127.0.0.1:42266)
0811 22:15:11.655516 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.655532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:11.655636 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:11.656050 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:11.656054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:11.656071 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:11.656172 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.656179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657411 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.657415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.658957 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.658970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.660252 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.660256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.660399 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.660424 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:15:11.660538 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:15:11.660638 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":32,"thread_start_us":119,"threads_started":1}
I20260811 22:15:11.660877 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.655110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.17.198.2:41433)
0811 22:15:11.655415 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:11.655431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:11.655608 (+ 177us) server_negotiation.cc:407] Connection header received
0811 22:15:11.655733 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:11.655736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:11.655796 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:11.655930 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:11.656316 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657246 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.659160 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.659931 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.660041 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.660127 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:11.660577 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 22:15:11.660675 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260811 22:15:11.660902 26290 tablet_service.cc:1467] Processing CreateTablet for tablet aa1488f059384d35ba3879afe66f6bd5 (DEFAULT_TABLE table=test-table [id=42c31219725844e0953192cf838c4181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:11.661286 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1488f059384d35ba3879afe66f6bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:11.661903 26140 tablet_service.cc:1467] Processing CreateTablet for tablet aa1488f059384d35ba3879afe66f6bd5 (DEFAULT_TABLE table=test-table [id=42c31219725844e0953192cf838c4181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:11.662200 26140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1488f059384d35ba3879afe66f6bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:11.664674 26490 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Bootstrap starting.
I20260811 22:15:11.665745 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.655192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41559 (local address 127.0.0.1:47318)
0811 22:15:11.655847 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.655861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:11.655948 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:11.656416 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:11.656419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:11.656439 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:11.656542 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.656554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657586 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.657591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.659018 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:11.659030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.661533 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.661537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:11.661630 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.665220 (+ 3590us) client_negotiation.cc:769] Sending connection context
0811 22:15:11.665449 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 22:15:11.665587 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":27,"thread_start_us":97,"threads_started":1}
I20260811 22:15:11.665884 26491 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Bootstrap starting.
I20260811 22:15:11.665966 26490 tablet_bootstrap.cc:654] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:11.666334 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.655366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.17.198.1:41559)
0811 22:15:11.655677 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:11.655684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:11.656088 (+ 404us) server_negotiation.cc:407] Connection header received
0811 22:15:11.656153 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:11.656156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:11.656207 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:11.656291 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:11.656685 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.657464 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.659191 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.661233 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.661379 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.661454 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:11.666075 (+ 4621us) server_negotiation.cc:299] Negotiation successful
0811 22:15:11.666215 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":124,"threads_started":1}
I20260811 22:15:11.667037 26491 tablet_bootstrap.cc:654] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:11.667119 25990 tablet_service.cc:1467] Processing CreateTablet for tablet aa1488f059384d35ba3879afe66f6bd5 (DEFAULT_TABLE table=test-table [id=42c31219725844e0953192cf838c4181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:11.667461 25990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1488f059384d35ba3879afe66f6bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:11.667972 26490 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: No bootstrap required, opened a new log
I20260811 22:15:11.668140 26490 ts_tablet_manager.cc:1397] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 22:15:11.668526 26491 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: No bootstrap required, opened a new log
I20260811 22:15:11.668630 26491 ts_tablet_manager.cc:1397] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:11.668717 26490 raft_consensus.cc:348] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.668926 26490 raft_consensus.cc:374] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:11.668996 26490 raft_consensus.cc:729] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c7e0ee92954ec2ab62caaa0442aff8, State: Initialized, Role: FOLLOWER
I20260811 22:15:11.669209 26490 consensus_queue.cc:260] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.669344 26491 raft_consensus.cc:348] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.669492 26491 raft_consensus.cc:374] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:11.669538 26491 raft_consensus.cc:729] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79968fcee6cb448d9856748434cd2600, State: Initialized, Role: FOLLOWER
I20260811 22:15:11.669713 26491 consensus_queue.cc:260] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.670300 26491 ts_tablet_manager.cc:1428] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:15:11.670113 26490 ts_tablet_manager.cc:1428] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:15:11.671332 26496 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: Bootstrap starting.
I20260811 22:15:11.672046 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.671660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:44293)
0811 22:15:11.671857 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.671949 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 22:15:11.672716 26496 tablet_bootstrap.cc:654] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:11.674907 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.673959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:53243)
0811 22:15:11.674520 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.674650 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427}
I20260811 22:15:11.675943 26215 heartbeater.cc:503] Master 127.17.198.61:45713 was elected leader, sending a full tablet report...
I20260811 22:15:11.676393 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.675951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:53369)
0811 22:15:11.676130 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.676240 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:15:11.677794 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.677420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:45147)
0811 22:15:11.677618 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.677675 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 22:15:11.678418 26496 tablet_bootstrap.cc:492] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: No bootstrap required, opened a new log
I20260811 22:15:11.678491 26368 heartbeater.cc:503] Master 127.17.198.61:45713 was elected leader, sending a full tablet report...
I20260811 22:15:11.678570 26496 ts_tablet_manager.cc:1397] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:15:11.679406 26496 raft_consensus.cc:348] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.679673 26496 raft_consensus.cc:374] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:11.679771 26496 raft_consensus.cc:729] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a6303526d44853b8761ef8c98f3699, State: Initialized, Role: FOLLOWER
I20260811 22:15:11.680145 26496 consensus_queue.cc:260] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.681306 26496 ts_tablet_manager.cc:1428] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:15:11.681488 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.681199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:35583)
0811 22:15:11.681320 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.681407 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 22:15:11.682361 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.682022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:59695)
0811 22:15:11.682196 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.682277 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:15:11.683996 26061 heartbeater.cc:503] Master 127.17.198.61:45713 was elected leader, sending a full tablet report...
I20260811 22:15:11.819346 26498 raft_consensus.cc:482] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:11.819566 26498 raft_consensus.cc:504] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.820106 26498 leader_election.cc:288] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213), 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
I20260811 22:15:11.820674 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1488f059384d35ba3879afe66f6bd5" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" is_pre_election: true
I20260811 22:15:11.820662 26166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1488f059384d35ba3879afe66f6bd5" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79968fcee6cb448d9856748434cd2600" is_pre_election: true
I20260811 22:15:11.820894 26166 raft_consensus.cc:2454] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 0.
I20260811 22:15:11.820904 26315 raft_consensus.cc:2454] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 0.
I20260811 22:15:11.821285 25940 leader_election.cc:302] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a6303526d44853b8761ef8c98f3699, 79968fcee6cb448d9856748434cd2600; no voters:
I20260811 22:15:11.821575 26498 raft_consensus.cc:2781] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:11.821653 26498 raft_consensus.cc:482] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:11.821691 26498 raft_consensus.cc:3037] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:11.823112 26498 raft_consensus.cc:504] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.823547 26498 leader_election.cc:288] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 election: Requested vote from peers b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213), 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433)
W20260811 22:15:11.823524 26216 tablet.cc:2369] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:11.824049 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1488f059384d35ba3879afe66f6bd5" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8"
I20260811 22:15:11.824194 26315 raft_consensus.cc:3037] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:11.824234 26166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1488f059384d35ba3879afe66f6bd5" candidate_uuid: "01a6303526d44853b8761ef8c98f3699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79968fcee6cb448d9856748434cd2600"
I20260811 22:15:11.824369 26166 raft_consensus.cc:3037] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:11.825460 26315 raft_consensus.cc:2454] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 1.
I20260811 22:15:11.825562 26166 raft_consensus.cc:2454] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a6303526d44853b8761ef8c98f3699 in term 1.
I20260811 22:15:11.825909 25940 leader_election.cc:302] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a6303526d44853b8761ef8c98f3699, b7c7e0ee92954ec2ab62caaa0442aff8; no voters:
I20260811 22:15:11.826133 26498 raft_consensus.cc:2781] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:11.826380 26498 raft_consensus.cc:686] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 1 LEADER]: Becoming Leader. State: Replica: 01a6303526d44853b8761ef8c98f3699, State: Running, Role: LEADER
I20260811 22:15:11.826607 26498 consensus_queue.cc:237] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } }
I20260811 22:15:11.827316 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.827047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:50347)
0811 22:15:11.827168 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.827240 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 22:15:11.828895 26369 tablet.cc:2369] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:11.828776 25654 catalog_manager.cc:5495] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a6303526d44853b8761ef8c98f3699 (127.17.198.1). New cstate: current_term: 1 leader_uuid: "01a6303526d44853b8761ef8c98f3699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a6303526d44853b8761ef8c98f3699" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41559 } health_report { overall_health: HEALTHY } } }
W20260811 22:15:11.841125 26062 tablet.cc:2369] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:11.858994 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.855053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.17.198.1:41559)
0811 22:15:11.855207 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:11.855211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:11.855228 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:11.855339 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:11.855342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:11.855393 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:11.855476 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:11.855798 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.856529 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.857712 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.858035 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.858143 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.858448 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:11.858539 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:11.858605 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:11.858742 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:15:11.858806 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:15:11.861958 26315 raft_consensus.cc:1261] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Refusing update from remote peer 01a6303526d44853b8761ef8c98f3699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:11.862128 26166 raft_consensus.cc:1261] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Refusing update from remote peer 01a6303526d44853b8761ef8c98f3699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:11.862818 26499 consensus_queue.cc:1035] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7c7e0ee92954ec2ab62caaa0442aff8" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:11.863076 26498 consensus_queue.cc:1035] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79968fcee6cb448d9856748434cd2600" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:11.865694 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.864953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:47637)
0811 22:15:11.865514 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.865584 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475}
I20260811 22:15:11.865948 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.865639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:52977)
0811 22:15:11.865781 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.865855 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:15:11.869666 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.869177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:45683)
0811 22:15:11.869458 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.869531 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 22:15:11.869757 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.869350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.2:52323)
0811 22:15:11.869589 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.869651 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 22:15:11.869984 26504 mvcc.cc:205] Tried to move back new op lower bound from 7317448752580325376 to 7317448752443453440. Current Snapshot: MvccSnapshot[applied={T|T < 7317448752580325376}]
I20260811 22:15:11.870239 26503 mvcc.cc:205] Tried to move back new op lower bound from 7317448752580325376 to 7317448752443453440. Current Snapshot: MvccSnapshot[applied={T|T < 7317448752580325376}]
I20260811 22:15:11.872983 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.872525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.1:48577)
0811 22:15:11.872778 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.872893 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 22:15:11.874805 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.874376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42627 (local address 127.17.198.3:60629)
0811 22:15:11.874621 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.874696 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:42627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 22:15:11.876515 25654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:11.876672 25654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:11.881780 25654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50e69706d13b44fab124875994d52d40: Sending DeleteTablet for 3 replicas of tablet aa1488f059384d35ba3879afe66f6bd5
I20260811 22:15:11.882767 26290 tablet_service.cc:1514] Processing DeleteTablet for tablet aa1488f059384d35ba3879afe66f6bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:11 UTC) from {username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:49266
I20260811 22:15:11.882848 26140 tablet_service.cc:1514] Processing DeleteTablet for tablet aa1488f059384d35ba3879afe66f6bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:11 UTC) from {username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:42266
I20260811 22:15:11.883244 25990 tablet_service.cc:1514] Processing DeleteTablet for tablet aa1488f059384d35ba3879afe66f6bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:11 UTC) from {username='oryx', principal='oryx/127.17.198.61@KRBTEST.COM'} at 127.0.0.1:47318
I20260811 22:15:11.883319 26515 tablet_replica.cc:330] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: stopping tablet replica
I20260811 22:15:11.883467 26515 raft_consensus.cc:2229] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:11.883672 26517 tablet_replica.cc:330] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699: stopping tablet replica
I20260811 22:15:11.883708 26516 tablet_replica.cc:330] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: stopping tablet replica
I20260811 22:15:11.883744 26515 raft_consensus.cc:2258] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:11.883877 26517 raft_consensus.cc:2229] T aa1488f059384d35ba3879afe66f6bd5 P 01a6303526d44853b8761ef8c98f3699 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:11.884039 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25817
I20260811 22:15:11.883847 26516 raft_consensus.cc:2229] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:11.885083 26516 raft_consensus.cc:2258] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:11.886716 26516 ts_tablet_manager.cc:1905] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:11.889178 26515 ts_tablet_manager.cc:1905] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:11.889784 26516 ts_tablet_manager.cc:1918] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:11.889905 26516 log.cc:1196] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/aa1488f059384d35ba3879afe66f6bd5
I20260811 22:15:11.890250 26516 ts_tablet_manager.cc:1939] T aa1488f059384d35ba3879afe66f6bd5 P 79968fcee6cb448d9856748434cd2600: Deleting consensus metadata
I20260811 22:15:11.891377 25636 catalog_manager.cc:4841] TS 79968fcee6cb448d9856748434cd2600 (127.17.198.2:41433): tablet aa1488f059384d35ba3879afe66f6bd5 (table test-table [id=42c31219725844e0953192cf838c4181]) successfully deleted
I20260811 22:15:11.893251 26515 ts_tablet_manager.cc:1918] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:11.893360 26515 log.cc:1196] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/aa1488f059384d35ba3879afe66f6bd5
I20260811 22:15:11.893688 26515 ts_tablet_manager.cc:1939] T aa1488f059384d35ba3879afe66f6bd5 P b7c7e0ee92954ec2ab62caaa0442aff8: Deleting consensus metadata
I20260811 22:15:11.895167 25636 catalog_manager.cc:4841] TS b7c7e0ee92954ec2ab62caaa0442aff8 (127.17.198.3:42213): tablet aa1488f059384d35ba3879afe66f6bd5 (table test-table [id=42c31219725844e0953192cf838c4181]) successfully deleted
W20260811 22:15:11.897734 26485 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:11.897337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41559 (local address 127.0.0.1:47334)
0811 22:15:11.897550 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:11.897615 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41559: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 22:15:11.897780 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26069
W20260811 22:15:11.901999 25637 catalog_manager.cc:4584] TS 01a6303526d44853b8761ef8c98f3699 (127.17.198.1:41559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa1488f059384d35ba3879afe66f6bd5: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41559: connect: Connection reset by peer (error 104)
I20260811 22:15:11.907588 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26221
I20260811 22:15:11.918282 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25504
I20260811 22:15:11.929668 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 25612
2026-08-11T22:15:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7698 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:12 dist-test-slave-77v9 krb5kdc[26521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:12 dist-test-slave-77v9 krb5kdc[26521](info): set up 2 sockets
Aug 11 22:15:12 dist-test-slave-77v9 krb5kdc[26521](info): commencing operation
krb5kdc: starting...
W20260811 22:15:13.123811 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:13 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab.
Entry for principal oryx/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab.
I20260811 22:15:13.274539 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42419
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab
--principal=oryx/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:13.397185 26537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:13.397569 26537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:13.397648 26537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:13.402215 26537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:13.402321 26537 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:15:13.402351 26537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:13.402386 26537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:13.402437 26537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:13.402474 26537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:13.408910 26537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42419
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.62.keytab
--principal=oryx/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:13.410266 26537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:13.411432 26537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:13.421140 26543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:13.421340 26537 server_base.cc:1034] running on GCE node
W20260811 22:15:13.421205 26546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:13.421140 26544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:13.422274 26537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:13.423635 26537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:13.424923 26537 hybrid_clock.cc:648] HybridClock initialized: now 1786486513424902 us; error 38 us; skew 500 ppm
Aug 11 22:15:13 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:13.433190 26537 init.cc:377] Logged in from keytab as oryx/127.17.198.62@KRBTEST.COM (short username oryx)
I20260811 22:15:13.434515 26537 webserver.cc:460] Webserver started at http://127.17.198.62:46367/ using document root <none> and password file <none>
I20260811 22:15:13.435285 26537 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:13.435366 26537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:13.435626 26537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:13.439041 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "0046a92d189045d190d3c48096dc5bf1"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "9aaf60a057097478f71be0097de67da8"
server_key_iv: "c7c7b3eeacdb1bcd528828723e78a4b7"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.440014 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "0046a92d189045d190d3c48096dc5bf1"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "9aaf60a057097478f71be0097de67da8"
server_key_iv: "c7c7b3eeacdb1bcd528828723e78a4b7"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.446151 26537 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:15:13.449239 26554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:13.450943 26537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:15:13.451155 26537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "0046a92d189045d190d3c48096dc5bf1"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "9aaf60a057097478f71be0097de67da8"
server_key_iv: "c7c7b3eeacdb1bcd528828723e78a4b7"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.451350 26537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:13.467618 26537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:13.472069 26537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:13.472354 26537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:13.482645 26537 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:42419
I20260811 22:15:13.482690 26632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:42419 every 8 connection(s)
I20260811 22:15:13.483786 26537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:13.487156 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:13.488680 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26537
I20260811 22:15:13.488924 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:13.489269 18200 external_mini_cluster.cc:1428] Setting key b0854a8a7d235e52dd31ca2357cc5782
I20260811 22:15:13.494021 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Bootstrap starting.
I20260811 22:15:13.496548 26633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:13.497624 26633 log.cc:824] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:15:13 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:13.500435 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: No bootstrap required, opened a new log
I20260811 22:15:13.504042 26633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } }
I20260811 22:15:13.504350 26633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:13.504428 26633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0046a92d189045d190d3c48096dc5bf1, State: Initialized, Role: FOLLOWER
I20260811 22:15:13.504987 26633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } }
I20260811 22:15:13.505189 26633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:13.505326 26633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:13.505450 26633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:13.506590 26633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } }
I20260811 22:15:13.506744 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:13.490885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.17.198.62:42419)
0811 22:15:13.491390 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:13.491407 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:13.491456 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:15:13.492186 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:13.492205 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:13.492570 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:13.493046 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:13.493910 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.495001 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.495898 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.496406 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.496592 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:13.500157 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:13.500163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:13.500178 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:13.500209 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:13.502859 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.503469 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.503476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.503482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.503612 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.504016 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.504020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.504022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.504640 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:13.504936 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:13.505249 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:15:13.505863 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":153,"threads_started":1}
I20260811 22:15:13.507027 26633 leader_election.cc:302] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0046a92d189045d190d3c48096dc5bf1; no voters:
I20260811 22:15:13.507438 26633 leader_election.cc:288] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:13.507575 26639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:13.507774 26639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [term 1 LEADER]: Becoming Leader. State: Replica: 0046a92d189045d190d3c48096dc5bf1, State: Running, Role: LEADER
I20260811 22:15:13.508217 26639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } }
I20260811 22:15:13.508955 26633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:13.510298 26640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0046a92d189045d190d3c48096dc5bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } } }
I20260811 22:15:13.510309 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0046a92d189045d190d3c48096dc5bf1. Latest consensus state: current_term: 1 leader_uuid: "0046a92d189045d190d3c48096dc5bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0046a92d189045d190d3c48096dc5bf1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42419 } } }
I20260811 22:15:13.510480 26640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:13.510484 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:13.510891 26650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:13.514184 26650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:13.522121 26650 catalog_manager.cc:1347] Generated new cluster ID: 54daf90fc8184100ae7166da641d159c
I20260811 22:15:13.522238 26650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:13.552198 26650 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:13.553704 26650 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:13.569881 26650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Generated new TSK 0
I20260811 22:15:13.570879 26650 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
I20260811 22:15:13.613952 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42419
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:13.743942 26665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:13.744248 26665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:13.744305 26665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:13.748725 26665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:13.748842 26665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:13.748886 26665 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:15:13.748972 26665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:13.754762 26665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:13.756052 26665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:13.757238 26665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:13.768417 26675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:13.768378 26674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:13.768395 26677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:13.768678 26665 server_base.cc:1034] running on GCE node
I20260811 22:15:13.769248 26665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:13.769992 26665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:13.771230 26665 hybrid_clock.cc:648] HybridClock initialized: now 1786486513771206 us; error 31 us; skew 500 ppm
Aug 11 22:15:13 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:13.774924 26665 init.cc:377] Logged in from keytab as oryx/127.17.198.1@KRBTEST.COM (short username oryx)
I20260811 22:15:13.776157 26665 webserver.cc:460] Webserver started at http://127.17.198.1:44645/ using document root <none> and password file <none>
I20260811 22:15:13.776970 26665 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:13.777092 26665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:13.777377 26665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:13.779989 26665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "ea08de68fb404b568997c2217963d529"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "135e67a67c46f85935b744bc3541cd6d"
server_key_iv: "88af4ceeffa927c2248032b83561b31a"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.780748 26665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "ea08de68fb404b568997c2217963d529"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "135e67a67c46f85935b744bc3541cd6d"
server_key_iv: "88af4ceeffa927c2248032b83561b31a"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.786057 26665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:15:13.789503 26685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:13.791083 26665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:13.791312 26665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "ea08de68fb404b568997c2217963d529"
format_stamp: "Formatted at 2026-08-11 22:15:13 on dist-test-slave-77v9"
server_key: "135e67a67c46f85935b744bc3541cd6d"
server_key_iv: "88af4ceeffa927c2248032b83561b31a"
server_key_version: "encryptionkey@0"
I20260811 22:15:13.791505 26665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:13.805308 26665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:13.809551 26665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:13.809825 26665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:13.811934 26665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:13.812028 26665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:13.812103 26665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:13.812161 26665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:15:13 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:13.824637 26665 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:36853
I20260811 22:15:13.824669 26817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:36853 every 8 connection(s)
I20260811 22:15:13.826076 26665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:13.828507 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:13.812847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:34747 (local address 127.17.198.62:42419)
0811 22:15:13.812996 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:13.812999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:13.814286 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 22:15:13.815538 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:13.815545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:13.815619 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:13.815694 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:13.817346 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.818318 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.820046 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.820314 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.820460 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:13.823756 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:13.823764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:13.823767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:13.823799 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:13.825724 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.826582 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.826589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.826594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.826720 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.827218 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.827222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.827223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.827457 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:13.827550 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:13.828188 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 22:15:13.828360 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:15:13.829420 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:13.813542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.1:34747)
0811 22:15:13.814104 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:13.814153 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:13.815196 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:13.816075 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:13.816082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:13.816473 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:13.817154 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:13.817177 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.818489 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.818494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:13.819874 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:13.819889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.820521 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.820526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:13.820735 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:13.821555 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:13.821573 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:13.823484 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:13.825931 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:13.825938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:13.825953 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:13.826407 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:13.826850 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:13.826854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:13.826856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:13.827061 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:13.827574 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:13.827580 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:13.827867 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 22:15:13.828040 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:15:13.828369 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":145,"threads_started":1}
I20260811 22:15:13.831005 26818 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42419
I20260811 22:15:13.831182 26818 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:13.831919 26818 heartbeater.cc:511] Master 127.17.198.62:42419 requested a full tablet report, sending...
I20260811 22:15:13.834084 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26665
I20260811 22:15:13.834134 26569 ts_manager.cc:194] Registered new tserver with Master: ea08de68fb404b568997c2217963d529 (127.17.198.1:36853)
I20260811 22:15:13.834241 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:13.834553 18200 external_mini_cluster.cc:1428] Setting key 39744d8c566cd2731f9d6e961f6be747
I20260811 22:15:13.835889 26569 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:34747
I20260811 22:15:13.843883 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:13.833785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:36229 (local address 127.17.198.62:42419)
0811 22:15:13.834001 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:13.834007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:13.835227 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 22:15:13.835362 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:13.835366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:13.835418 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:13.835519 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:13.836221 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.837111 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.838115 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.838395 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.838534 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:13.839998 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:13.840002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:13.840005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:13.840046 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:13.841820 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.842417 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.842421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.842423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.842527 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:13.842853 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:13.842858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:13.842860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:13.843078 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:13.843210 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:13.843535 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:15:13.843680 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:15:13.843890 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:13.833704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.1:36229)
0811 22:15:13.835090 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:13.835104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:13.835254 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:13.835561 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:13.835564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:13.835834 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:13.835996 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:13.836004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.837285 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.837289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:13.837968 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:13.837977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:13.838543 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:13.838547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:13.838702 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:13.839378 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:13.839391 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:13.839798 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:13.842048 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:13.842051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:13.842053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:13.842295 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:13.842636 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:13.842641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:13.842643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:13.842745 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:13.843195 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:13.843199 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:13.843319 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:15:13.843444 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:15:13.843648 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1284,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for oryx/127.17.198.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
I20260811 22:15:13.874337 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42419
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:14.004534 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:14.004901 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:14.004958 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:14.009423 26829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:14.009512 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:14.009618 26829 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:15:14.009703 26829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:14.016358 26829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:14.017750 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:14.018962 26829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:15:14.030462 26829 server_base.cc:1034] running on GCE node
W20260811 22:15:14.030406 26839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.030419 26836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.030372 26835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:14.031329 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:14.032220 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:14.033551 26829 hybrid_clock.cc:648] HybridClock initialized: now 1786486514033463 us; error 108 us; skew 500 ppm
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:14.037148 26829 init.cc:377] Logged in from keytab as oryx/127.17.198.2@KRBTEST.COM (short username oryx)
I20260811 22:15:14.038341 26829 webserver.cc:460] Webserver started at http://127.17.198.2:38043/ using document root <none> and password file <none>
I20260811 22:15:14.039041 26829 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:14.039161 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:14.039425 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:14.041956 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "c7418c52dc894e7ba7414c4e062aa987"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "2d18fb6e8f68989d8a47087c8cb8a50e"
server_key_iv: "f11388fd02c549a29332b025b556bd9c"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.042645 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "c7418c52dc894e7ba7414c4e062aa987"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "2d18fb6e8f68989d8a47087c8cb8a50e"
server_key_iv: "f11388fd02c549a29332b025b556bd9c"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.047468 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:14.050774 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.052268 26829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:14.052452 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "c7418c52dc894e7ba7414c4e062aa987"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "2d18fb6e8f68989d8a47087c8cb8a50e"
server_key_iv: "f11388fd02c549a29332b025b556bd9c"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.052608 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:14.073326 26829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:14.077518 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:14.077786 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:14.079547 26829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:14.079638 26829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.079706 26829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:14.079758 26829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:14.091711 26829 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:38369
I20260811 22:15:14.091902 26975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:38369 every 8 connection(s)
I20260811 22:15:14.092746 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:14.094228 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26829
I20260811 22:15:14.094362 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:14.094719 18200 external_mini_cluster.cc:1428] Setting key 0732d144a542b2b7a06d2256a6928f24
I20260811 22:15:14.098664 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.081135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.2:35697)
0811 22:15:14.081696 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.081776 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.083051 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.083918 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.083926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.084516 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.085395 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.085417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.086461 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.086465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.087882 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.087894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.088619 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.088628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.088874 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.089659 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.089686 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.091884 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.094289 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.094310 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.094332 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.094826 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.095366 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.095369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.095371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.095625 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.096258 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.096266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.096693 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.096953 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.097375 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260811 22:15:14.099601 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.080623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:35697 (local address 127.17.198.62:42419)
0811 22:15:14.080842 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.080848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.081939 (+ 1091us) server_negotiation.cc:407] Connection header received
0811 22:15:14.083430 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.083435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.083521 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.083641 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.085597 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.086293 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.088053 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.088354 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.088510 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.092171 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.092175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.092177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.092208 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.094144 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.095045 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.095050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.095052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.095195 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.095781 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.095784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.095786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.096077 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.096182 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.099294 (+ 3112us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.099478 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:15:14.101241 26976 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42419
I20260811 22:15:14.101438 26976 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:14.102126 26976 heartbeater.cc:511] Master 127.17.198.62:42419 requested a full tablet report, sending...
I20260811 22:15:14.103515 26574 ts_manager.cc:194] Registered new tserver with Master: c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:14.104532 26574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:35697
WARNING: no policy specified for oryx/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.3@KRBTEST.COM" created.
I20260811 22:15:14.113365 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.103759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.2:54385)
0811 22:15:14.105109 (+ 1350us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.105126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.105261 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.105543 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.105546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.105804 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.105910 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.105917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.106732 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.106736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.107357 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.107367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.107850 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.107854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.107949 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.108574 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.108586 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.109065 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.111324 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.111328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.111330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.111657 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.111979 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.111982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.111984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.112081 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.112705 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.112708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.112803 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.113030 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.113181 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":113,"threads_started":1}
I20260811 22:15:14.113555 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.103763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:54385 (local address 127.17.198.62:42419)
0811 22:15:14.103915 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.103920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.105213 (+ 1293us) server_negotiation.cc:407] Connection header received
0811 22:15:14.105380 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.105383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.105437 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.105527 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.106038 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.106600 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.107484 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.107726 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.107842 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.109213 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.109217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.109219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.109246 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.111169 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.111779 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.111783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.111785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.111887 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.112179 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.112183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.112185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.112581 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.112699 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.113121 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.113308 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
I20260811 22:15:14.134366 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42419
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:14.261355 26986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:14.261752 26986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:14.261842 26986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:14.266386 26986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:14.266541 26986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:14.266623 26986 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:15:14.266712 26986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:14.272684 26986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.198.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:14.274051 26986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:14.275208 26986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:14.286159 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.286196 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.286160 26995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:14.286581 26986 server_base.cc:1034] running on GCE node
I20260811 22:15:14.287156 26986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:14.287959 26986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:14.289186 26986 hybrid_clock.cc:648] HybridClock initialized: now 1786486514289151 us; error 52 us; skew 500 ppm
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:14.292470 26986 init.cc:377] Logged in from keytab as oryx/127.17.198.3@KRBTEST.COM (short username oryx)
I20260811 22:15:14.293596 26986 webserver.cc:460] Webserver started at http://127.17.198.3:43665/ using document root <none> and password file <none>
I20260811 22:15:14.294250 26986 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:14.294329 26986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:14.294571 26986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:14.297266 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "67d90cdab3794a2da162f3eccc62794c"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "f2e6a43bb0463650673c5a2a3d8dfd34"
server_key_iv: "c9f1b22c3f21a4a4f5089b8a36f47912"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.298028 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "67d90cdab3794a2da162f3eccc62794c"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "f2e6a43bb0463650673c5a2a3d8dfd34"
server_key_iv: "c9f1b22c3f21a4a4f5089b8a36f47912"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.302838 26986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:14.305986 27003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.307483 26986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:14.307703 26986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "67d90cdab3794a2da162f3eccc62794c"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-77v9"
server_key: "f2e6a43bb0463650673c5a2a3d8dfd34"
server_key_iv: "c9f1b22c3f21a4a4f5089b8a36f47912"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.307855 26986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:14.324214 26986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:14.328236 26986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:14.328538 26986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:14.330477 26986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:14.330547 26986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.330617 26986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:14.330647 26986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:14.343058 26986 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:45171
I20260811 22:15:14.343401 27136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:45171 every 8 connection(s)
I20260811 22:15:14.344521 26986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:14.346793 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.331367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47815 (local address 127.17.198.62:42419)
0811 22:15:14.331602 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.331611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.332560 (+ 949us) server_negotiation.cc:407] Connection header received
0811 22:15:14.333793 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.333798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.333875 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.333989 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.335985 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.336627 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.338014 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.338444 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.338584 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.341866 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.341874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.341877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.341915 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.344092 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.344971 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.344979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.344984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.345119 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.345631 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.345634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.345635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.345876 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.345970 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.346523 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.346657 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:15:14.347779 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.331892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.3:47815)
0811 22:15:14.332368 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.332423 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.333467 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.334377 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.334391 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.334927 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.335766 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.335791 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.336786 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.336789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.337862 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.337876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.338657 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.338662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.338910 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.339681 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.339698 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.341582 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.344288 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.344298 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.344319 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.344756 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.345256 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.345260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.345263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.345483 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.345991 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.345996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.346254 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.346423 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.346761 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":132,"threads_started":1}
I20260811 22:15:14.349431 27137 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42419
I20260811 22:15:14.349617 27137 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:14.350308 27137 heartbeater.cc:511] Master 127.17.198.62:42419 requested a full tablet report, sending...
I20260811 22:15:14.351711 26569 ts_manager.cc:194] Registered new tserver with Master: 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171)
I20260811 22:15:14.352694 26569 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:47815
I20260811 22:15:14.353979 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 26986
I20260811 22:15:14.354144 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:14.354511 18200 external_mini_cluster.cc:1428] Setting key d8cc8e119a6c1c7a4d16700017a7d71e
I20260811 22:15:14.357476 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:14.361471 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.352217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.17.198.3:54455)
0811 22:15:14.352576 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.352590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.352741 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.353189 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.353192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.353405 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.353512 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.353517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.354520 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.354523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.355514 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.355529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.356147 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.356151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.356245 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.356979 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.356999 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.357378 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.359481 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.359484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.359487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.359722 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.360097 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.360104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.360109 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.360252 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.360804 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.360860 (+ 56us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.361017 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.361123 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.361286 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":140,"threads_started":1}
I20260811 22:15:14.361613 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.352347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:54455 (local address 127.17.198.62:42419)
0811 22:15:14.352529 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.352534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.352672 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:15:14.352865 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.352868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.352920 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.353009 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.353686 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.354357 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.355627 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.355931 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.356189 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.357556 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.357560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.357562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.357621 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.359322 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.359842 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.359845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.359847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.359941 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.360392 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.360405 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.360407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.360689 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.360834 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.361219 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.361415 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:15:14.387432 26636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:14.379965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.17.198.62:42419)
0811 22:15:14.380154 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.380158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.380173 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:14.380219 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.380221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.380262 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.380343 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.381316 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.381912 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.382867 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.383132 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.383224 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.387293 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38282: BlockingRecv error: recv got EOF from 127.0.0.1:38282 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:14.400198 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.389675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.17.198.62:42419)
0811 22:15:14.389843 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.389847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.390089 (+ 242us) server_negotiation.cc:407] Connection header received
0811 22:15:14.390216 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.390219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.390267 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.390352 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.391105 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.391701 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.392728 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.393087 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.393208 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.396273 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.396277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.396279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.396312 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.398046 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.398591 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.398595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.398596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.398882 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.399221 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.399223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.399225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.399444 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.399556 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.399941 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.400067 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:15:14.403970 26569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:14.407425 26569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:14.425025 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.418515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.17.198.1:36853)
0811 22:15:14.420231 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.420238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.420260 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:15:14.420332 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.420337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.420523 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.420673 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.420950 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.421954 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.423237 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.424089 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.424214 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.424396 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.424727 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.424875 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1587,"thread_start_us":110,"threads_started":1}
I20260811 22:15:14.425180 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.418089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36853 (local address 127.0.0.1:36470)
0811 22:15:14.418706 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.418779 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.418943 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.420677 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.420680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.420700 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.420805 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.420842 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.422094 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.422099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.423101 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.423111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.424234 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.424237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.424412 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.424485 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.424620 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.424855 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":47,"spinlock_wait_cycles":12032,"thread_start_us":110,"threads_started":1}
I20260811 22:15:14.425181 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.417760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:38369 (local address 127.0.0.1:46488)
0811 22:15:14.418708 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.418779 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.418941 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.419914 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.419917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.419935 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.420105 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.420112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.421610 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.421613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.423307 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.423318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.424361 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.424364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.424486 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.424505 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.424620 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.424855 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":207,"spinlock_wait_cycles":9344,"thread_start_us":111,"threads_started":1}
I20260811 22:15:14.427124 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.417995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.17.198.2:38369)
0811 22:15:14.419350 (+ 1355us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.419366 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.419392 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:15:14.419507 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.419516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.419729 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.419927 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.420257 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.421467 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.423463 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.424232 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.424335 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.424544 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.426735 (+ 2191us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.426963 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":138,"threads_started":1}
I20260811 22:15:14.427652 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.418417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.17.198.3:45171)
0811 22:15:14.419303 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.419316 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.419355 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:15:14.419434 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.419439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.419675 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.419843 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.420244 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.421277 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.424690 (+ 3413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.425651 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.425774 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.426038 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.426092 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.427527 (+ 1435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":143,"threads_started":1}
I20260811 22:15:14.429575 26741 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba (DEFAULT_TABLE table=test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.430953 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e24530fa47fbb6aca10ec2b203ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.426247 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.417766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:45171 (local address 127.0.0.1:34764)
0811 22:15:14.419201 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.419216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.419302 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.419858 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.419868 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.419898 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.420103 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.420112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.421397 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.421404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.423306 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.423318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.425791 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.425795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.425896 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.425921 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.426009 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.426106 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"mutex_wait_us":1125,"thread_start_us":104,"threads_started":1}
I20260811 22:15:14.433935 26902 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba (DEFAULT_TABLE table=test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.434916 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e24530fa47fbb6aca10ec2b203ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.441547 27172 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Bootstrap starting.
I20260811 22:15:14.441766 27065 tablet_service.cc:1467] Processing CreateTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba (DEFAULT_TABLE table=test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.443315 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e2e24530fa47fbb6aca10ec2b203ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.444670 27172 tablet_bootstrap.cc:654] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.445676 27172 log.cc:824] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:14.447079 27173 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Bootstrap starting.
I20260811 22:15:14.449893 27173 tablet_bootstrap.cc:654] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.451007 27173 log.cc:824] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:14.451153 27172 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: No bootstrap required, opened a new log
I20260811 22:15:14.451396 27172 ts_tablet_manager.cc:1397] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 22:15:14.454139 27173 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: No bootstrap required, opened a new log
I20260811 22:15:14.454149 27176 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Bootstrap starting.
I20260811 22:15:14.454547 27173 ts_tablet_manager.cc:1397] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:15:14.457240 27176 tablet_bootstrap.cc:654] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.458379 27176 log.cc:824] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:14.458483 27172 raft_consensus.cc:348] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.458838 27172 raft_consensus.cc:374] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.458904 27172 raft_consensus.cc:729] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08de68fb404b568997c2217963d529, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.459101 27173 raft_consensus.cc:348] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.459378 27173 raft_consensus.cc:374] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.459435 27173 raft_consensus.cc:729] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418c52dc894e7ba7414c4e062aa987, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.459508 27172 consensus_queue.cc:260] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.459838 27173 consensus_queue.cc:260] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.460326 26818 heartbeater.cc:503] Master 127.17.198.62:42419 was elected leader, sending a full tablet report...
I20260811 22:15:14.460956 27173 ts_tablet_manager.cc:1428] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:15:14.461320 26976 heartbeater.cc:503] Master 127.17.198.62:42419 was elected leader, sending a full tablet report...
I20260811 22:15:14.461949 27176 tablet_bootstrap.cc:492] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: No bootstrap required, opened a new log
I20260811 22:15:14.462239 27172 ts_tablet_manager.cc:1428] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Time spent starting tablet: real 0.011s user 0.004s sys 0.005s
I20260811 22:15:14.462241 27176 ts_tablet_manager.cc:1397] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:15:14.467314 27176 raft_consensus.cc:348] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.467720 27176 raft_consensus.cc:374] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.467801 27176 raft_consensus.cc:729] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.468377 27176 consensus_queue.cc:260] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.469640 27137 heartbeater.cc:503] Master 127.17.198.62:42419 was elected leader, sending a full tablet report...
I20260811 22:15:14.470121 27176 ts_tablet_manager.cc:1428] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260811 22:15:14.475754 27180 raft_consensus.cc:482] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:14.475957 27180 raft_consensus.cc:504] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.477339 27180 leader_election.cc:288] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369), ea08de68fb404b568997c2217963d529 (127.17.198.1:36853)
I20260811 22:15:14.482498 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.478154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:40771 (local address 127.17.198.2:38369)
0811 22:15:14.478311 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.478315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.478330 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:14.478382 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.478384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.478463 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.478550 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.479104 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.479607 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.480902 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.481548 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.481700 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.481759 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.482268 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.482368 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:15:14.482498 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.477808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:38369 (local address 127.17.198.3:40771)
0811 22:15:14.477933 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.477952 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.478075 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.478696 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.478700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.478720 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.478957 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.478969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.479765 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.479776 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.480704 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.480716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.481825 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.481830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.482081 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.482103 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.482239 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.482343 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":72,"spinlock_wait_cycles":18560}
I20260811 22:15:14.483434 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.477898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:38597 (local address 127.17.198.1:36853)
0811 22:15:14.478078 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.478082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.478468 (+ 386us) server_negotiation.cc:407] Connection header received
0811 22:15:14.478560 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.478563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.478614 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.478702 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.479005 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.479591 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.481911 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.482623 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.482754 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.482812 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.483217 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.483312 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:15:14.483645 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e24530fa47fbb6aca10ec2b203ba" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987" is_pre_election: true
I20260811 22:15:14.484009 26925 raft_consensus.cc:2454] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:14.484375 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.477808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36853 (local address 127.17.198.3:38597)
0811 22:15:14.478349 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.478361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.478468 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.478719 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.478721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.478735 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.478880 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.478885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.479724 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.479731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.481762 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.481776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.482951 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.482954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.483051 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.483074 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.483185 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.483255 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":136,"thread_start_us":149,"threads_started":1}
I20260811 22:15:14.484578 27008 leader_election.cc:302] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, c7418c52dc894e7ba7414c4e062aa987; no voters:
I20260811 22:15:14.484927 27180 raft_consensus.cc:2781] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:14.485049 27180 raft_consensus.cc:482] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:14.485105 27180 raft_consensus.cc:3037] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.485037 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e24530fa47fbb6aca10ec2b203ba" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529" is_pre_election: true
I20260811 22:15:14.485339 26765 raft_consensus.cc:2454] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:14.486529 27180 raft_consensus.cc:504] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.487001 27180 leader_election.cc:288] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Requested vote from peers c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369), ea08de68fb404b568997c2217963d529 (127.17.198.1:36853)
I20260811 22:15:14.487373 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e24530fa47fbb6aca10ec2b203ba" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987"
I20260811 22:15:14.487531 26925 raft_consensus.cc:3037] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.487633 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e2e24530fa47fbb6aca10ec2b203ba" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529"
I20260811 22:15:14.487763 26765 raft_consensus.cc:3037] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.489017 26765 raft_consensus.cc:2454] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:14.489018 26925 raft_consensus.cc:2454] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:14.489409 27007 leader_election.cc:302] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:14.489602 27180 raft_consensus.cc:2781] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:14.489853 27180 raft_consensus.cc:686] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Becoming Leader. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Running, Role: LEADER
I20260811 22:15:14.490234 27180 consensus_queue.cc:237] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } }
I20260811 22:15:14.493700 26574 catalog_manager.cc:5495] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c reported cstate change: term changed from 0 to 1, leader changed from <none> to 67d90cdab3794a2da162f3eccc62794c (127.17.198.3). New cstate: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } health_report { overall_health: UNKNOWN } } }
I20260811 22:15:14.514708 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.510773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.17.198.3:45171)
0811 22:15:14.510903 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.510908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.510931 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:15:14.511020 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.511022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.511087 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.511182 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:14.511547 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.512121 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.513146 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.513414 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.513510 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.513815 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:14.514303 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:14.514392 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.514494 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.514553 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:15:14.521399 26925 raft_consensus.cc:1261] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:14.521391 26765 raft_consensus.cc:1261] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:14.522220 27180 consensus_queue.cc:1035] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:14.522462 27183 consensus_queue.cc:1035] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:14.531064 27188 mvcc.cc:205] Tried to move back new op lower bound from 7317448763470872576 to 7317448763354894336. Current Snapshot: MvccSnapshot[applied={T|T < 7317448763470872576}]
I20260811 22:15:14.531905 27186 mvcc.cc:205] Tried to move back new op lower bound from 7317448763470872576 to 7317448763354894336. Current Snapshot: MvccSnapshot[applied={T|T < 7317448763470872576}]
I20260811 22:15:14.541290 26569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:14.541484 26569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:14.544498 26569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Sending DeleteTablet for 3 replicas of tablet 71e2e24530fa47fbb6aca10ec2b203ba
I20260811 22:15:14.545542 27065 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:14 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:34764
I20260811 22:15:14.545552 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:14 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46488
I20260811 22:15:14.545749 26741 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e2e24530fa47fbb6aca10ec2b203ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:14 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36470
I20260811 22:15:14.546190 27196 tablet_replica.cc:330] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: stopping tablet replica
I20260811 22:15:14.546231 27197 tablet_replica.cc:330] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: stopping tablet replica
I20260811 22:15:14.546588 27197 raft_consensus.cc:2229] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:14.546588 27196 raft_consensus.cc:2229] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:14.546682 26569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:15:14.546931 27196 raft_consensus.cc:2258] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:14.546972 27197 raft_consensus.cc:2258] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:15:14.547176 26569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:14.548604 27197 ts_tablet_manager.cc:1905] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:14.548604 27196 ts_tablet_manager.cc:1905] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:14.549510 27198 tablet_replica.cc:330] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: stopping tablet replica
I20260811 22:15:14.550004 27198 raft_consensus.cc:2229] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:14.550511 27198 raft_consensus.cc:2258] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:14.553727 27198 ts_tablet_manager.cc:1905] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:14.554872 26741 tablet_service.cc:1467] Processing CreateTablet for tablet ec160fb22ead473b8ea4f92322e66699 (DEFAULT_TABLE table=test-table [id=5815ff72211f46a6a2fc89af9f629c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.555205 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec160fb22ead473b8ea4f92322e66699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.555279 26902 tablet_service.cc:1467] Processing CreateTablet for tablet ec160fb22ead473b8ea4f92322e66699 (DEFAULT_TABLE table=test-table [id=5815ff72211f46a6a2fc89af9f629c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.555590 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec160fb22ead473b8ea4f92322e66699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.557101 27198 ts_tablet_manager.cc:1918] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:14.557296 27198 log.cc:1196] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/71e2e24530fa47fbb6aca10ec2b203ba
I20260811 22:15:14.557811 27172 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Bootstrap starting.
I20260811 22:15:14.558149 27173 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Bootstrap starting.
I20260811 22:15:14.558295 27197 ts_tablet_manager.cc:1918] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:14.558430 27197 log.cc:1196] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/71e2e24530fa47fbb6aca10ec2b203ba
I20260811 22:15:14.558403 27065 tablet_service.cc:1467] Processing CreateTablet for tablet ec160fb22ead473b8ea4f92322e66699 (DEFAULT_TABLE table=test-table [id=5815ff72211f46a6a2fc89af9f629c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:14.558688 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec160fb22ead473b8ea4f92322e66699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.558789 27197 ts_tablet_manager.cc:1939] T 71e2e24530fa47fbb6aca10ec2b203ba P 67d90cdab3794a2da162f3eccc62794c: Deleting consensus metadata
I20260811 22:15:14.559175 27173 tablet_bootstrap.cc:654] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.559307 27172 tablet_bootstrap.cc:654] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.559998 27198 ts_tablet_manager.cc:1939] T 71e2e24530fa47fbb6aca10ec2b203ba P ea08de68fb404b568997c2217963d529: Deleting consensus metadata
I20260811 22:15:14.561081 26555 catalog_manager.cc:4841] TS 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171): tablet 71e2e24530fa47fbb6aca10ec2b203ba (table test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]) successfully deleted
I20260811 22:15:14.561184 27196 ts_tablet_manager.cc:1918] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:14.561290 27196 log.cc:1196] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/71e2e24530fa47fbb6aca10ec2b203ba
I20260811 22:15:14.561671 27196 ts_tablet_manager.cc:1939] T 71e2e24530fa47fbb6aca10ec2b203ba P c7418c52dc894e7ba7414c4e062aa987: Deleting consensus metadata
I20260811 22:15:14.562023 26557 catalog_manager.cc:4841] TS ea08de68fb404b568997c2217963d529 (127.17.198.1:36853): tablet 71e2e24530fa47fbb6aca10ec2b203ba (table test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]) successfully deleted
I20260811 22:15:14.562458 27172 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: No bootstrap required, opened a new log
I20260811 22:15:14.562562 27172 ts_tablet_manager.cc:1397] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 22:15:14.563076 26559 catalog_manager.cc:4841] TS c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369): tablet 71e2e24530fa47fbb6aca10ec2b203ba (table test-table [id=0ba1d1c66c94434d8f45d7b1a9237287]) successfully deleted
I20260811 22:15:14.563069 27172 raft_consensus.cc:348] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.563192 27172 raft_consensus.cc:374] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.563223 27172 raft_consensus.cc:729] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08de68fb404b568997c2217963d529, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.563360 27172 consensus_queue.cc:260] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.563618 27172 ts_tablet_manager.cc:1428] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:14.563889 27176 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Bootstrap starting.
I20260811 22:15:14.564713 27173 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: No bootstrap required, opened a new log
I20260811 22:15:14.564864 27173 ts_tablet_manager.cc:1397] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:15:14.565182 27176 tablet_bootstrap.cc:654] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.565459 27173 raft_consensus.cc:348] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.565613 27173 raft_consensus.cc:374] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.565663 27173 raft_consensus.cc:729] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418c52dc894e7ba7414c4e062aa987, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.565821 27173 consensus_queue.cc:260] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.566143 27173 ts_tablet_manager.cc:1428] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:14.567694 27176 tablet_bootstrap.cc:492] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: No bootstrap required, opened a new log
I20260811 22:15:14.567837 27176 ts_tablet_manager.cc:1397] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:15:14.568369 27176 raft_consensus.cc:348] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.568540 27176 raft_consensus.cc:374] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.568590 27176 raft_consensus.cc:729] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.568734 27176 consensus_queue.cc:260] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.569133 27176 ts_tablet_manager.cc:1428] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260811 22:15:14.578222 26819 tablet.cc:2369] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:15:14.594260 26977 tablet.cc:2369] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:15:14.595715 27138 tablet.cc:2369] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:14.742847 27178 raft_consensus.cc:482] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:14.743034 27178 raft_consensus.cc:504] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.744215 27178 leader_election.cc:288] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369), 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171)
I20260811 22:15:14.748749 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.744580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:48737 (local address 127.17.198.2:38369)
0811 22:15:14.744754 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.744760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.745164 (+ 404us) server_negotiation.cc:407] Connection header received
0811 22:15:14.745362 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.745365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.745409 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.745501 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.745811 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.746329 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.747644 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.748089 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.748239 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.748316 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.748532 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.748606 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:15:14.748775 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.744475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:38369 (local address 127.17.198.1:48737)
0811 22:15:14.745023 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.745042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.745220 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.745517 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.745521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.745537 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.745678 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.745683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.746446 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.746452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.747499 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.747510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.748240 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.748244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.748402 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.748420 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.748512 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.748592 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":104,"thread_start_us":117,"threads_started":1}
I20260811 22:15:14.748760 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.744842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:45277 (local address 127.17.198.3:45171)
0811 22:15:14.745026 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.745032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.745226 (+ 194us) server_negotiation.cc:407] Connection header received
0811 22:15:14.745435 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.745438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.745494 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.745586 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:14.745873 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.746423 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.747653 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.748082 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.748239 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.748305 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.748556 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.748653 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:15:14.749303 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.744696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:45171 (local address 127.17.198.1:45277)
0811 22:15:14.745140 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.745155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.745262 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.745601 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.745604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.745621 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:14.745750 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.745755 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.746571 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.746577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.747516 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.747526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.748240 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.748244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.748379 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.748399 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.749084 (+ 685us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.749163 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":22,"thread_start_us":153,"threads_started":1}
I20260811 22:15:14.749406 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec160fb22ead473b8ea4f92322e66699" candidate_uuid: "ea08de68fb404b568997c2217963d529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987" is_pre_election: true
I20260811 22:15:14.749631 26925 raft_consensus.cc:2454] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea08de68fb404b568997c2217963d529 in term 0.
I20260811 22:15:14.749993 27087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec160fb22ead473b8ea4f92322e66699" candidate_uuid: "ea08de68fb404b568997c2217963d529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d90cdab3794a2da162f3eccc62794c" is_pre_election: true
I20260811 22:15:14.750144 26690 leader_election.cc:302] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7418c52dc894e7ba7414c4e062aa987, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:14.750241 27087 raft_consensus.cc:2454] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea08de68fb404b568997c2217963d529 in term 0.
I20260811 22:15:14.750469 27178 raft_consensus.cc:2781] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:14.750547 27178 raft_consensus.cc:482] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:14.750591 27178 raft_consensus.cc:3037] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.751689 27178 raft_consensus.cc:504] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.752023 27178 leader_election.cc:288] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [CANDIDATE]: Term 1 election: Requested vote from peers c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369), 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171)
I20260811 22:15:14.752646 27087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec160fb22ead473b8ea4f92322e66699" candidate_uuid: "ea08de68fb404b568997c2217963d529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d90cdab3794a2da162f3eccc62794c"
I20260811 22:15:14.752687 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec160fb22ead473b8ea4f92322e66699" candidate_uuid: "ea08de68fb404b568997c2217963d529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987"
I20260811 22:15:14.752800 27087 raft_consensus.cc:3037] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.752882 26925 raft_consensus.cc:3037] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.753995 26925 raft_consensus.cc:2454] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea08de68fb404b568997c2217963d529 in term 1.
I20260811 22:15:14.754017 27087 raft_consensus.cc:2454] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea08de68fb404b568997c2217963d529 in term 1.
I20260811 22:15:14.754380 26690 leader_election.cc:302] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, c7418c52dc894e7ba7414c4e062aa987, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:14.754673 27178 raft_consensus.cc:2781] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:14.754995 27178 raft_consensus.cc:686] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 1 LEADER]: Becoming Leader. State: Replica: ea08de68fb404b568997c2217963d529, State: Running, Role: LEADER
I20260811 22:15:14.755395 27178 consensus_queue.cc:237] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.758729 26569 catalog_manager.cc:5495] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea08de68fb404b568997c2217963d529 (127.17.198.1). New cstate: current_term: 1 leader_uuid: "ea08de68fb404b568997c2217963d529" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:15:14 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.62@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:14.856328 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.846831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.17.198.62:42419)
0811 22:15:14.846983 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.846988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.847163 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:15:14.847344 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.847346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.847381 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.847468 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.848168 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.848686 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.849578 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.849861 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.849997 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.852629 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.852635 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.852642 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.852663 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.854147 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.854653 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.854658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.854660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.854741 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.855045 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.855049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.855052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.855249 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.855383 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.855997 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.856152 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:15:14.856489 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.846747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42419 (local address 127.0.0.1:38290)
0811 22:15:14.847070 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.847083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.847227 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.847509 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.847512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.847950 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.848048 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.848055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.848804 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.848835 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.849458 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.849467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.850014 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.850017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.850097 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.850807 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.850825 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.852478 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.854268 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.854282 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.854290 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.854547 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.854835 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.854839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.854841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.854922 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.855383 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.855397 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.855610 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.855933 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.856237 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":87,"thread_start_us":113,"threads_started":1}
I20260811 22:15:14.861218 26569 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:38290:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:15:14.862116 26569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:14.867966 26902 tablet_service.cc:1467] Processing CreateTablet for tablet 2695679d005b48eda98a6de92c841b5e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc5e1f46f5a444f90c5c0e583d3869d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:15:14.868047 27065 tablet_service.cc:1467] Processing CreateTablet for tablet 2695679d005b48eda98a6de92c841b5e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc5e1f46f5a444f90c5c0e583d3869d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:15:14.868044 26741 tablet_service.cc:1467] Processing CreateTablet for tablet 2695679d005b48eda98a6de92c841b5e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbc5e1f46f5a444f90c5c0e583d3869d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:15:14.868454 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2695679d005b48eda98a6de92c841b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.868485 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2695679d005b48eda98a6de92c841b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.868485 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2695679d005b48eda98a6de92c841b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.871115 27173 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: Bootstrap starting.
I20260811 22:15:14.871973 27172 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: Bootstrap starting.
I20260811 22:15:14.871965 27176 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: Bootstrap starting.
I20260811 22:15:14.872259 27173 tablet_bootstrap.cc:654] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.872999 27176 tablet_bootstrap.cc:654] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.872991 27172 tablet_bootstrap.cc:654] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.873857 27173 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: No bootstrap required, opened a new log
I20260811 22:15:14.873991 27173 ts_tablet_manager.cc:1397] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 22:15:14.874404 27176 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: No bootstrap required, opened a new log
I20260811 22:15:14.874531 27176 ts_tablet_manager.cc:1397] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:14.874513 27173 raft_consensus.cc:348] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.874666 27173 raft_consensus.cc:374] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.874716 27173 raft_consensus.cc:729] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418c52dc894e7ba7414c4e062aa987, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.874857 27173 consensus_queue.cc:260] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.875077 27176 raft_consensus.cc:348] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.875166 27173 ts_tablet_manager.cc:1428] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:15:14.875224 27176 raft_consensus.cc:374] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.875276 27176 raft_consensus.cc:729] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.875418 27176 consensus_queue.cc:260] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.876000 27179 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:14.876142 27172 tablet_bootstrap.cc:492] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: No bootstrap required, opened a new log
I20260811 22:15:14.876192 27179 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:14.876243 27172 ts_tablet_manager.cc:1397] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 22:15:14.876425 27193 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:14.876571 27193 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:14.876730 27172 raft_consensus.cc:348] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.876896 27172 raft_consensus.cc:374] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.876955 27172 raft_consensus.cc:729] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08de68fb404b568997c2217963d529, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.876987 27176 ts_tablet_manager.cc:1428] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:15:14.877127 27172 consensus_queue.cc:260] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:14.877439 27172 ts_tablet_manager.cc:1428] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:15:14.877372 27178 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:14.877508 27178 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:15.049193 27193 raft_consensus.cc:482] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:15.049371 27193 raft_consensus.cc:504] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.050030 27193 leader_election.cc:288] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.050561 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2695679d005b48eda98a6de92c841b5e" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529" is_pre_election: true
I20260811 22:15:15.050570 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2695679d005b48eda98a6de92c841b5e" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987" is_pre_election: true
I20260811 22:15:15.050750 26765 raft_consensus.cc:2454] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.050761 26925 raft_consensus.cc:2454] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.051155 27007 leader_election.cc:302] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:15.051388 27193 raft_consensus.cc:2781] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:15.051463 27193 raft_consensus.cc:482] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:15.051505 27193 raft_consensus.cc:3037] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.052582 27193 raft_consensus.cc:504] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.053027 27193 leader_election.cc:288] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Requested vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.053468 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2695679d005b48eda98a6de92c841b5e" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529"
I20260811 22:15:15.053558 26925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2695679d005b48eda98a6de92c841b5e" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987"
I20260811 22:15:15.053629 26765 raft_consensus.cc:3037] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.053725 26925 raft_consensus.cc:3037] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.054649 26765 raft_consensus.cc:2454] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.054801 26925 raft_consensus.cc:2454] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.055181 27007 leader_election.cc:302] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:15.055492 27193 raft_consensus.cc:2781] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:15.055732 27193 raft_consensus.cc:686] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Becoming Leader. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Running, Role: LEADER
I20260811 22:15:15.055918 27193 consensus_queue.cc:237] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.057237 27216 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: TxnStatusTablet state changed. Reason: New leader 67d90cdab3794a2da162f3eccc62794c. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.057374 27216 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:15:15.057686 27218 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:15:15.057911 27218 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:15:15.058279 26575 catalog_manager.cc:5495] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c reported cstate change: term changed from 0 to 1, leader changed from <none> to 67d90cdab3794a2da162f3eccc62794c (127.17.198.3). New cstate: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260811 22:15:15.081395 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.077163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.17.198.3:45171)
0811 22:15:15.077304 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.077310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.077379 (+ 69us) server_negotiation.cc:407] Connection header received
0811 22:15:15.077736 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.077739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.077795 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.077896 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.078220 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.078833 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.080102 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.080339 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.080436 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.080894 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.080997 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.081073 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.081209 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.081270 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:15:15.081455 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.077070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:45171 (local address 127.0.0.1:34784)
0811 22:15:15.077289 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.077308 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.077471 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.077942 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.077945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.077953 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:15:15.078075 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.078081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.078958 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.078962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.079965 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.079977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.080472 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.080476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.080574 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.080643 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.081093 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.081114 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.081189 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.081293 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:15:15.084451 26925 raft_consensus.cc:1261] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.084502 26765 raft_consensus.cc:1261] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.084980 27193 consensus_queue.cc:1035] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.085143 27216 consensus_queue.cc:1035] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.086678 27178 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: TxnStatusTablet state changed. Reason: New leader 67d90cdab3794a2da162f3eccc62794c. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.086822 27178 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:15.088007 27179 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: TxnStatusTablet state changed. Reason: New leader 67d90cdab3794a2da162f3eccc62794c. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.088142 27179 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:15.090831 27179 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.090982 27179 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P c7418c52dc894e7ba7414c4e062aa987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:15.091311 27178 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.091455 27178 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P ea08de68fb404b568997c2217963d529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:15:15.093351 27216 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.093482 27216 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:15:15.094201 27223 mvcc.cc:205] Tried to move back new op lower bound from 7317448765781385216 to 7317448765671346176. Current Snapshot: MvccSnapshot[applied={T|T < 7317448765781385216 or (T in {7317448765781385216})}]
I20260811 22:15:15.094592 27193 tablet_replica.cc:439] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } } }
I20260811 22:15:15.094718 27193 tablet_replica.cc:442] T 2695679d005b48eda98a6de92c841b5e P 67d90cdab3794a2da162f3eccc62794c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:15:15.109314 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.103262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.17.198.1:36853)
0811 22:15:15.103498 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.103502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.105143 (+ 1641us) server_negotiation.cc:407] Connection header received
0811 22:15:15.105366 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.105368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.105433 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.105588 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.105863 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.106418 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.107431 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.107675 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.107815 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.108127 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.108730 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.108906 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.108946 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.109011 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":90,"threads_started":1}
I20260811 22:15:15.125674 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.121310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:45171 (local address 127.17.198.1:40743)
0811 22:15:15.121687 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.121702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.121812 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.122140 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.122142 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.122150 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:15:15.122275 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.122280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.123372 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.123375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.124089 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.124099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.124761 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.124765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.124962 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.125024 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.125374 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.125380 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.125441 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.125525 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":136,"threads_started":1}
I20260811 22:15:15.125667 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.121423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:40743 (local address 127.17.198.3:45171)
0811 22:15:15.121578 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.121584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.121789 (+ 205us) server_negotiation.cc:407] Connection header received
0811 22:15:15.121903 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.121906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.121968 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.122090 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.122395 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.123226 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.124243 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.124616 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.124724 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.125170 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.125281 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.125357 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.125457 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.125524 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:15:15.134498 26925 raft_consensus.cc:1261] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Refusing update from remote peer ea08de68fb404b568997c2217963d529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.134725 27087 raft_consensus.cc:1261] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Refusing update from remote peer ea08de68fb404b568997c2217963d529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.135099 27205 consensus_queue.cc:1035] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.135855 27205 consensus_queue.cc:1035] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.161599 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.157151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36853 (local address 127.17.198.3:44223)
0811 22:15:15.157513 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.157530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.157644 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.158026 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.158029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.158036 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:15:15.158136 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.158142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.159187 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.159191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.160155 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.160165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.160678 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.160682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.160776 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.160877 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.161233 (+ 356us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.161240 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.161310 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.161404 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":150,"threads_started":1}
I20260811 22:15:15.161625 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.157243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:44223 (local address 127.17.198.1:36853)
0811 22:15:15.157397 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.157403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.157616 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:15:15.157777 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.157781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.157833 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.157880 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.158248 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.159047 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.160286 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.160544 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.160644 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.161033 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.161117 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.161210 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.161322 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.161429 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:15:15.196568 26574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.196769 26574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.199506 26574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Sending DeleteTablet for 3 replicas of tablet ec160fb22ead473b8ea4f92322e66699
I20260811 22:15:15.200263 26741 tablet_service.cc:1514] Processing DeleteTablet for tablet ec160fb22ead473b8ea4f92322e66699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36470
I20260811 22:15:15.200567 27065 tablet_service.cc:1514] Processing DeleteTablet for tablet ec160fb22ead473b8ea4f92322e66699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:34764
I20260811 22:15:15.201177 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet ec160fb22ead473b8ea4f92322e66699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46488
I20260811 22:15:15.201638 27251 tablet_replica.cc:330] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: stopping tablet replica
I20260811 22:15:15.201861 27251 raft_consensus.cc:2229] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:15.202400 27251 raft_consensus.cc:2258] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.203300 27251 ts_tablet_manager.cc:1905] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.203593 27252 tablet_replica.cc:330] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: stopping tablet replica
I20260811 22:15:15.207067 27251 ts_tablet_manager.cc:1918] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:15:15.207343 27252 raft_consensus.cc:2229] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.207541 27252 raft_consensus.cc:2258] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.209060 27253 tablet_replica.cc:330] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: stopping tablet replica
I20260811 22:15:15.209229 27253 raft_consensus.cc:2229] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.209372 27253 raft_consensus.cc:2258] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.209637 27252 ts_tablet_manager.cc:1905] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.210182 27253 ts_tablet_manager.cc:1905] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.213168 27252 ts_tablet_manager.cc:1918] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:15:15.213363 27251 log.cc:1196] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/ec160fb22ead473b8ea4f92322e66699
I20260811 22:15:15.213550 27253 ts_tablet_manager.cc:1918] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:15:15.213716 27253 log.cc:1196] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/ec160fb22ead473b8ea4f92322e66699
I20260811 22:15:15.213954 27251 ts_tablet_manager.cc:1939] T ec160fb22ead473b8ea4f92322e66699 P ea08de68fb404b568997c2217963d529: Deleting consensus metadata
I20260811 22:15:15.214038 27253 ts_tablet_manager.cc:1939] T ec160fb22ead473b8ea4f92322e66699 P c7418c52dc894e7ba7414c4e062aa987: Deleting consensus metadata
I20260811 22:15:15.214941 26557 catalog_manager.cc:4841] TS ea08de68fb404b568997c2217963d529 (127.17.198.1:36853): tablet ec160fb22ead473b8ea4f92322e66699 (table test-table [id=5815ff72211f46a6a2fc89af9f629c8c]) successfully deleted
I20260811 22:15:15.214936 26559 catalog_manager.cc:4841] TS c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369): tablet ec160fb22ead473b8ea4f92322e66699 (table test-table [id=5815ff72211f46a6a2fc89af9f629c8c]) successfully deleted
I20260811 22:15:15.215950 27252 log.cc:1196] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/ec160fb22ead473b8ea4f92322e66699
I20260811 22:15:15.216272 27252 ts_tablet_manager.cc:1939] T ec160fb22ead473b8ea4f92322e66699 P 67d90cdab3794a2da162f3eccc62794c: Deleting consensus metadata
I20260811 22:15:15.217212 26555 catalog_manager.cc:4841] TS 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171): tablet ec160fb22ead473b8ea4f92322e66699 (table test-table [id=5815ff72211f46a6a2fc89af9f629c8c]) successfully deleted
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:15:15.251549 26636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:15.242930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.17.198.62:42419)
0811 22:15:15.243076 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.243081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.243404 (+ 323us) server_negotiation.cc:407] Connection header received
0811 22:15:15.243613 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.243617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.243672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.243768 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:15.244692 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.245546 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.246818 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.247172 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.247247 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.251401 (+ 4154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38302: BlockingRecv error: recv got EOF from 127.0.0.1:38302 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM
I20260811 22:15:15.264191 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.253591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.17.198.62:42419)
0811 22:15:15.253745 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.253757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.253906 (+ 149us) server_negotiation.cc:407] Connection header received
0811 22:15:15.254044 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.254046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.254094 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.254174 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:15.255085 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.255655 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.257011 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.257236 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.257367 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.260492 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:15.260496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:15.260498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:15.260532 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:15.262210 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.262745 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.262747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.262749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.262832 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.263195 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.263198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.263200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.263439 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:15.263560 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.263857 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.263995 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:15:15.267180 26574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:15.267681 26574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:15.272774 26741 tablet_service.cc:1467] Processing CreateTablet for tablet 9f94896faabb4f9183a1287031de7de5 (DEFAULT_TABLE table=test-table [id=9fb11250d1fe4eaa8a529336f1c27742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.272774 26902 tablet_service.cc:1467] Processing CreateTablet for tablet 9f94896faabb4f9183a1287031de7de5 (DEFAULT_TABLE table=test-table [id=9fb11250d1fe4eaa8a529336f1c27742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.273185 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f94896faabb4f9183a1287031de7de5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.273185 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f94896faabb4f9183a1287031de7de5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.276120 27173 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Bootstrap starting.
I20260811 22:15:15.276309 27172 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Bootstrap starting.
I20260811 22:15:15.276499 27065 tablet_service.cc:1467] Processing CreateTablet for tablet 9f94896faabb4f9183a1287031de7de5 (DEFAULT_TABLE table=test-table [id=9fb11250d1fe4eaa8a529336f1c27742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.276858 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f94896faabb4f9183a1287031de7de5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.277267 27173 tablet_bootstrap.cc:654] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.277400 27172 tablet_bootstrap.cc:654] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.278899 27172 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: No bootstrap required, opened a new log
I20260811 22:15:15.279006 27172 ts_tablet_manager.cc:1397] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:15:15.279556 27172 raft_consensus.cc:348] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.279795 27173 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: No bootstrap required, opened a new log
I20260811 22:15:15.279886 27173 ts_tablet_manager.cc:1397] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 22:15:15.279692 27172 raft_consensus.cc:374] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.279970 27172 raft_consensus.cc:729] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08de68fb404b568997c2217963d529, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.280185 27172 consensus_queue.cc:260] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.280340 27173 raft_consensus.cc:348] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.280447 27173 raft_consensus.cc:374] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.280478 27173 raft_consensus.cc:729] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418c52dc894e7ba7414c4e062aa987, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.280536 27172 ts_tablet_manager.cc:1428] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:15:15.280598 27173 consensus_queue.cc:260] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.281515 27173 ts_tablet_manager.cc:1428] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:15:15.282480 27176 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Bootstrap starting.
I20260811 22:15:15.283550 27176 tablet_bootstrap.cc:654] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.285465 27176 tablet_bootstrap.cc:492] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: No bootstrap required, opened a new log
I20260811 22:15:15.285594 27176 ts_tablet_manager.cc:1397] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:15.286051 27176 raft_consensus.cc:348] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.286188 27176 raft_consensus.cc:374] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.286231 27176 raft_consensus.cc:729] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.286382 27176 consensus_queue.cc:260] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.286690 27176 ts_tablet_manager.cc:1428] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:15:15.289703 27193 raft_consensus.cc:482] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:15.289830 27193 raft_consensus.cc:504] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.290246 27193 leader_election.cc:288] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.290678 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f94896faabb4f9183a1287031de7de5" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529" is_pre_election: true
I20260811 22:15:15.290838 26765 raft_consensus.cc:2454] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.291112 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f94896faabb4f9183a1287031de7de5" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987" is_pre_election: true
I20260811 22:15:15.291162 27007 leader_election.cc:302] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:15.291265 26924 raft_consensus.cc:2454] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.291404 27193 raft_consensus.cc:2781] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:15.291494 27193 raft_consensus.cc:482] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:15.291554 27193 raft_consensus.cc:3037] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.292894 27193 raft_consensus.cc:504] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.293335 27193 leader_election.cc:288] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Requested vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.293676 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f94896faabb4f9183a1287031de7de5" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529"
I20260811 22:15:15.293835 26765 raft_consensus.cc:3037] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.293783 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f94896faabb4f9183a1287031de7de5" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987"
I20260811 22:15:15.293958 26924 raft_consensus.cc:3037] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.294968 26765 raft_consensus.cc:2454] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.295287 27007 leader_election.cc:302] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:15.295514 27193 raft_consensus.cc:2781] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:15.295626 27193 raft_consensus.cc:686] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Becoming Leader. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Running, Role: LEADER
I20260811 22:15:15.295610 26924 raft_consensus.cc:2454] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.295780 27193 consensus_queue.cc:237] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.297683 26574 catalog_manager.cc:5495] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c reported cstate change: term changed from 0 to 1, leader changed from <none> to 67d90cdab3794a2da162f3eccc62794c (127.17.198.3). New cstate: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260811 22:15:15.309702 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.305761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.17.198.3:45171)
0811 22:15:15.305901 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.305905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.305919 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:15:15.306060 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.306062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.306107 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.306172 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.306457 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.307000 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.308241 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.308467 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.308550 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.309000 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.309115 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.309208 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.309390 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.309466 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:15:15.312152 26765 raft_consensus.cc:1261] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.312271 26924 raft_consensus.cc:1261] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.312665 27193 consensus_queue.cc:1035] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.312845 27233 consensus_queue.cc:1035] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.322510 26574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.322667 26574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.324663 26574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Sending DeleteTablet for 3 replicas of tablet 9f94896faabb4f9183a1287031de7de5
I20260811 22:15:15.325309 26741 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f94896faabb4f9183a1287031de7de5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36470
I20260811 22:15:15.325510 27251 tablet_replica.cc:330] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: stopping tablet replica
I20260811 22:15:15.325482 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f94896faabb4f9183a1287031de7de5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46488
I20260811 22:15:15.325635 27251 raft_consensus.cc:2229] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.325677 27253 tablet_replica.cc:330] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: stopping tablet replica
I20260811 22:15:15.325788 27251 raft_consensus.cc:2258] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.325806 27253 raft_consensus.cc:2229] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.325951 27253 raft_consensus.cc:2258] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.325979 27065 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f94896faabb4f9183a1287031de7de5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:34764
I20260811 22:15:15.326179 27252 tablet_replica.cc:330] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: stopping tablet replica
I20260811 22:15:15.326323 27252 raft_consensus.cc:2229] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:15.326568 27252 raft_consensus.cc:2258] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.327247 27252 ts_tablet_manager.cc:1905] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.327247 27253 ts_tablet_manager.cc:1905] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.327416 27251 ts_tablet_manager.cc:1905] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.329803 27252 ts_tablet_manager.cc:1918] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:15.329803 27253 ts_tablet_manager.cc:1918] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:15.329887 27253 log.cc:1196] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/9f94896faabb4f9183a1287031de7de5
I20260811 22:15:15.329887 27252 log.cc:1196] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/9f94896faabb4f9183a1287031de7de5
I20260811 22:15:15.330123 27251 ts_tablet_manager.cc:1918] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:15.330169 27253 ts_tablet_manager.cc:1939] T 9f94896faabb4f9183a1287031de7de5 P c7418c52dc894e7ba7414c4e062aa987: Deleting consensus metadata
I20260811 22:15:15.330169 27252 ts_tablet_manager.cc:1939] T 9f94896faabb4f9183a1287031de7de5 P 67d90cdab3794a2da162f3eccc62794c: Deleting consensus metadata
I20260811 22:15:15.330199 27251 log.cc:1196] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/9f94896faabb4f9183a1287031de7de5
I20260811 22:15:15.331238 26559 catalog_manager.cc:4841] TS c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369): tablet 9f94896faabb4f9183a1287031de7de5 (table test-table [id=9fb11250d1fe4eaa8a529336f1c27742]) successfully deleted
I20260811 22:15:15.331445 27251 ts_tablet_manager.cc:1939] T 9f94896faabb4f9183a1287031de7de5 P ea08de68fb404b568997c2217963d529: Deleting consensus metadata
I20260811 22:15:15.331509 26574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:15.332141 26574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:15.332867 26555 catalog_manager.cc:4841] TS 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171): tablet 9f94896faabb4f9183a1287031de7de5 (table test-table [id=9fb11250d1fe4eaa8a529336f1c27742]) successfully deleted
I20260811 22:15:15.333346 26557 catalog_manager.cc:4841] TS ea08de68fb404b568997c2217963d529 (127.17.198.1:36853): tablet 9f94896faabb4f9183a1287031de7de5 (table test-table [id=9fb11250d1fe4eaa8a529336f1c27742]) successfully deleted
I20260811 22:15:15.337769 26741 tablet_service.cc:1467] Processing CreateTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f (DEFAULT_TABLE table=test-table [id=373707eb75304c31a93fc9a5285b5fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.337900 26902 tablet_service.cc:1467] Processing CreateTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f (DEFAULT_TABLE table=test-table [id=373707eb75304c31a93fc9a5285b5fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.338162 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e6fc7d75004718a66b32e5ee98fb9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.338313 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e6fc7d75004718a66b32e5ee98fb9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.340612 27173 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Bootstrap starting.
I20260811 22:15:15.341622 27065 tablet_service.cc:1467] Processing CreateTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f (DEFAULT_TABLE table=test-table [id=373707eb75304c31a93fc9a5285b5fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.341907 27173 tablet_bootstrap.cc:654] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.341988 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e6fc7d75004718a66b32e5ee98fb9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.343437 27173 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: No bootstrap required, opened a new log
I20260811 22:15:15.343609 27173 ts_tablet_manager.cc:1397] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:15.344206 27173 raft_consensus.cc:348] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.344384 27173 raft_consensus.cc:374] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.344427 27173 raft_consensus.cc:729] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418c52dc894e7ba7414c4e062aa987, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.344578 27173 consensus_queue.cc:260] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.345002 27173 ts_tablet_manager.cc:1428] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:15:15.349874 27176 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Bootstrap starting.
I20260811 22:15:15.349993 27172 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: Bootstrap starting.
I20260811 22:15:15.351052 27176 tablet_bootstrap.cc:654] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.351351 27172 tablet_bootstrap.cc:654] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.352761 27176 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: No bootstrap required, opened a new log
I20260811 22:15:15.352933 27176 ts_tablet_manager.cc:1397] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:15.353363 27176 raft_consensus.cc:348] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.353514 27176 raft_consensus.cc:374] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.353557 27176 raft_consensus.cc:729] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.353708 27176 consensus_queue.cc:260] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.354090 27176 ts_tablet_manager.cc:1428] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:15:15.355154 27172 tablet_bootstrap.cc:492] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: No bootstrap required, opened a new log
I20260811 22:15:15.355269 27172 ts_tablet_manager.cc:1397] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 22:15:15.355801 27172 raft_consensus.cc:348] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.355940 27172 raft_consensus.cc:374] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.355979 27172 raft_consensus.cc:729] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08de68fb404b568997c2217963d529, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.356164 27172 consensus_queue.cc:260] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.357046 27172 ts_tablet_manager.cc:1428] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 22:15:15.393134 27193 raft_consensus.cc:482] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:15.393313 27193 raft_consensus.cc:504] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.393774 27193 leader_election.cc:288] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.394302 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e6fc7d75004718a66b32e5ee98fb9f" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987" is_pre_election: true
I20260811 22:15:15.394330 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e6fc7d75004718a66b32e5ee98fb9f" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529" is_pre_election: true
I20260811 22:15:15.394520 26765 raft_consensus.cc:2454] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.394521 26924 raft_consensus.cc:2454] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 0.
I20260811 22:15:15.394924 27008 leader_election.cc:302] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, c7418c52dc894e7ba7414c4e062aa987; no voters:
I20260811 22:15:15.395146 27193 raft_consensus.cc:2781] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:15.395229 27193 raft_consensus.cc:482] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:15.395282 27193 raft_consensus.cc:3037] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.396500 27193 raft_consensus.cc:504] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.397096 27193 leader_election.cc:288] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Requested vote from peers ea08de68fb404b568997c2217963d529 (127.17.198.1:36853), c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369)
I20260811 22:15:15.397555 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e6fc7d75004718a66b32e5ee98fb9f" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea08de68fb404b568997c2217963d529"
I20260811 22:15:15.397720 26765 raft_consensus.cc:3037] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.398726 26765 raft_consensus.cc:2454] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.399145 27007 leader_election.cc:302] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d90cdab3794a2da162f3eccc62794c, ea08de68fb404b568997c2217963d529; no voters:
I20260811 22:15:15.399116 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e6fc7d75004718a66b32e5ee98fb9f" candidate_uuid: "67d90cdab3794a2da162f3eccc62794c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418c52dc894e7ba7414c4e062aa987"
I20260811 22:15:15.399271 26924 raft_consensus.cc:3037] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:15.399356 27193 raft_consensus.cc:2781] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:15.399470 27193 raft_consensus.cc:686] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Becoming Leader. State: Replica: 67d90cdab3794a2da162f3eccc62794c, State: Running, Role: LEADER
I20260811 22:15:15.399667 27193 consensus_queue.cc:237] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } }
I20260811 22:15:15.400578 26924 raft_consensus.cc:2454] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d90cdab3794a2da162f3eccc62794c in term 1.
I20260811 22:15:15.401823 26569 catalog_manager.cc:5495] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c reported cstate change: term changed from 0 to 1, leader changed from <none> to 67d90cdab3794a2da162f3eccc62794c (127.17.198.3). New cstate: current_term: 1 leader_uuid: "67d90cdab3794a2da162f3eccc62794c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d90cdab3794a2da162f3eccc62794c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260811 22:15:15.437914 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.434388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:51693 (local address 127.17.198.3:45171)
0811 22:15:15.434507 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.434512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.434578 (+ 66us) server_negotiation.cc:407] Connection header received
0811 22:15:15.434745 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.434747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.434789 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.434875 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:15.435107 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.435640 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.436515 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.436726 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.436872 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.437358 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.437446 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.437509 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.437675 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.437767 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":51}
I20260811 22:15:15.437942 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.434352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:45171 (local address 127.17.198.3:51693)
0811 22:15:15.434512 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.434528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.434636 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.434893 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.434896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.434902 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:15:15.435001 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.435007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.435759 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.435762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.436381 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.436391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.437143 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.437147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.437214 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.437244 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:15:15.437570 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:15:15.437577 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.437657 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.437803 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 22:15:15.445112 26765 raft_consensus.cc:1261] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.445112 26924 raft_consensus.cc:1261] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Refusing update from remote peer 67d90cdab3794a2da162f3eccc62794c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:15.445557 27228 consensus_queue.cc:1035] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea08de68fb404b568997c2217963d529" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.445741 27193 consensus_queue.cc:1035] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418c52dc894e7ba7414c4e062aa987" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:15.482300 26569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.482481 26569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:15.484783 26569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0046a92d189045d190d3c48096dc5bf1: Sending DeleteTablet for 3 replicas of tablet c6e6fc7d75004718a66b32e5ee98fb9f
I20260811 22:15:15.485728 27064 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:34764
I20260811 22:15:15.485780 26741 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36470
I20260811 22:15:15.485822 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e6fc7d75004718a66b32e5ee98fb9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:15 UTC) from {username='oryx', principal='oryx/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46488
I20260811 22:15:15.485967 27252 tablet_replica.cc:330] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: stopping tablet replica
I20260811 22:15:15.485989 27251 tablet_replica.cc:330] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529: stopping tablet replica
I20260811 22:15:15.486018 27253 tablet_replica.cc:330] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: stopping tablet replica
I20260811 22:15:15.486091 27252 raft_consensus.cc:2229] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:15.486110 27251 raft_consensus.cc:2229] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.486141 27253 raft_consensus.cc:2229] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:15.486244 27253 raft_consensus.cc:2258] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.486263 27252 raft_consensus.cc:2258] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.486241 27251 raft_consensus.cc:2258] T c6e6fc7d75004718a66b32e5ee98fb9f P ea08de68fb404b568997c2217963d529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:15.487207 27252 ts_tablet_manager.cc:1905] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.489316 27253 ts_tablet_manager.cc:1905] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:15.490032 27252 ts_tablet_manager.cc:1918] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:15:15.490115 27252 log.cc:1196] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/c6e6fc7d75004718a66b32e5ee98fb9f
I20260811 22:15:15.490372 27252 ts_tablet_manager.cc:1939] T c6e6fc7d75004718a66b32e5ee98fb9f P 67d90cdab3794a2da162f3eccc62794c: Deleting consensus metadata
I20260811 22:15:15.491223 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26665
I20260811 22:15:15.494326 27253 ts_tablet_manager.cc:1918] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:15:15.494450 27253 log.cc:1196] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/c6e6fc7d75004718a66b32e5ee98fb9f
I20260811 22:15:15.494716 27253 ts_tablet_manager.cc:1939] T c6e6fc7d75004718a66b32e5ee98fb9f P c7418c52dc894e7ba7414c4e062aa987: Deleting consensus metadata
I20260811 22:15:15.495666 26559 catalog_manager.cc:4841] TS c7418c52dc894e7ba7414c4e062aa987 (127.17.198.2:38369): tablet c6e6fc7d75004718a66b32e5ee98fb9f (table test-table [id=373707eb75304c31a93fc9a5285b5fc6]) successfully deleted
I20260811 22:15:15.496615 26555 catalog_manager.cc:4841] TS 67d90cdab3794a2da162f3eccc62794c (127.17.198.3:45171): tablet c6e6fc7d75004718a66b32e5ee98fb9f (table test-table [id=373707eb75304c31a93fc9a5285b5fc6]) successfully deleted
W20260811 22:15:15.505376 26557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:36853 (error 108)
I20260811 22:15:15.506716 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.506295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36853 (local address 127.0.0.1:36486)
0811 22:15:15.506535 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.506601 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
I20260811 22:15:15.506880 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26829
W20260811 22:15:15.507016 26557 catalog_manager.cc:4584] TS ea08de68fb404b568997c2217963d529 (127.17.198.1:36853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6e6fc7d75004718a66b32e5ee98fb9f: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36853: connect: Connection refused (error 111)
I20260811 22:15:15.518402 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26986
I20260811 22:15:15.529174 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 26537
2026-08-11T22:15:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3597 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[27292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[27292](info): set up 2 sockets
Aug 11 22:15:15 dist-test-slave-77v9 krb5kdc[27292](info): commencing operation
krb5kdc: starting...
W20260811 22:15:16.733217 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:16 dist-test-slave-77v9 krb5kdc[27292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:15:16.881589 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46609
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:46609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:17.002468 27306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:17.002831 27306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:17.002912 27306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:17.007229 27306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:17.007349 27306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:17.007397 27306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:17.007445 27306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:17.007501 27306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:17.013334 27306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:46609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:17.014618 27306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:17.015763 27306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:17.025420 27314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:17.025425 27317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:17.025468 27306 server_base.cc:1034] running on GCE node
W20260811 22:15:17.025425 27313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:17.026337 27306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:17.027805 27306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:17.028966 27306 hybrid_clock.cc:648] HybridClock initialized: now 1786486517028948 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:15:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1508 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:17 dist-test-slave-77v9 krb5kdc[27323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:17 dist-test-slave-77v9 krb5kdc[27323](info): set up 2 sockets
Aug 11 22:15:17 dist-test-slave-77v9 krb5kdc[27323](info): commencing operation
krb5kdc: starting...
W20260811 22:15:18.261637 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:18 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:15:18.409260 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39661
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:39661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:18.533574 27338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:18.533994 27338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:18.534092 27338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:18.538479 27338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:18.538633 27338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:18.538676 27338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:18.538719 27338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:18.538793 27338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:18.544683 27338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:39661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:18.546010 27338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:18.547168 27338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:18.557080 27346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.557094 27345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.557088 27349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:18.557989 27338 server_base.cc:1034] running on GCE node
I20260811 22:15:18.558656 27338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:18.560012 27338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:18.561223 27338 hybrid_clock.cc:648] HybridClock initialized: now 1786486518561193 us; error 47 us; skew 500 ppm
Aug 11 22:15:18 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:18.568172 27338 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:15:18.569347 27338 webserver.cc:460] Webserver started at http://127.17.198.62:42661/ using document root <none> and password file <none>
I20260811 22:15:18.569957 27338 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:18.570039 27338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:18.570235 27338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:18.572710 27338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "571ccd61cfad4a60aee27ca7333de0ae"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "5f008210a036f439e63f481bf1662d9a"
server_key_iv: "cc4f7962b247c3be74f1e6468d9f6618"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.573468 27338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "571ccd61cfad4a60aee27ca7333de0ae"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "5f008210a036f439e63f481bf1662d9a"
server_key_iv: "cc4f7962b247c3be74f1e6468d9f6618"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.578088 27338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:18.580943 27356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.582432 27338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:18.582574 27338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "571ccd61cfad4a60aee27ca7333de0ae"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "5f008210a036f439e63f481bf1662d9a"
server_key_iv: "cc4f7962b247c3be74f1e6468d9f6618"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.582690 27338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:18.610788 27338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:18.614444 27338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:18.614621 27338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:18.622709 27338 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:39661
I20260811 22:15:18.622738 27417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:39661 every 8 connection(s)
I20260811 22:15:18.623776 27338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:18.626823 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:18.628243 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 27338
I20260811 22:15:18.628419 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:18.628751 18200 external_mini_cluster.cc:1428] Setting key 752aa83a8a1cde13cc156231db4c07b0
I20260811 22:15:18.633677 27419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Bootstrap starting.
I20260811 22:15:18.636149 27419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:18.637032 27419 log.cc:824] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:15:18 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:18.639555 27419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: No bootstrap required, opened a new log
I20260811 22:15:18.643697 27419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:18.643962 27419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:18.643998 27419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571ccd61cfad4a60aee27ca7333de0ae, State: Initialized, Role: FOLLOWER
I20260811 22:15:18.644564 27419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:18.644737 27419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:18.644846 27419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:18.644986 27419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:18.646054 27419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:18.646159 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.630458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.17.198.62:39661)
0811 22:15:18.630984 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.631008 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.631100 (+ 92us) server_negotiation.cc:407] Connection header received
0811 22:15:18.631947 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.631967 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.632362 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.632748 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:18.633719 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.634883 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.635903 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.636530 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.636729 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.640052 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:18.640059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:18.640076 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:18.640108 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:18.642388 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.643012 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.643027 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.643033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.643173 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.643526 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.643531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.643533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.644111 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:18.644373 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.644707 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.645318 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":157,"threads_started":1}
I20260811 22:15:18.646546 27419 leader_election.cc:302] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 571ccd61cfad4a60aee27ca7333de0ae; no voters:
I20260811 22:15:18.646926 27419 leader_election.cc:288] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:18.647102 27424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:18.647377 27424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 LEADER]: Becoming Leader. State: Replica: 571ccd61cfad4a60aee27ca7333de0ae, State: Running, Role: LEADER
I20260811 22:15:18.647764 27424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:18.648259 27419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:18.650138 27425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "571ccd61cfad4a60aee27ca7333de0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } } }
I20260811 22:15:18.650343 27425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:18.650279 27426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 571ccd61cfad4a60aee27ca7333de0ae. Latest consensus state: current_term: 1 leader_uuid: "571ccd61cfad4a60aee27ca7333de0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } } }
I20260811 22:15:18.650414 27426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:18.650964 27434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:18.654461 27434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:18.662492 27434 catalog_manager.cc:1347] Generated new cluster ID: 347e47092ca14067ba13141ce4ddfd79
I20260811 22:15:18.662601 27434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:18.702368 27434 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:18.703601 27434 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:18.719350 27434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Generated new TSK 0
I20260811 22:15:18.720582 27434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:15:18.769636 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:18.888701 27448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:18.889086 27448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:18.889166 27448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:18.893373 27448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:18.893452 27448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:18.893513 27448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:18.899392 27448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:18.901088 27448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:18.902410 27448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:18.914247 27454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.914255 27457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.914738 27455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:18.914886 27448 server_base.cc:1034] running on GCE node
I20260811 22:15:18.915347 27448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:18.916100 27448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:18.917294 27448 hybrid_clock.cc:648] HybridClock initialized: now 1786486518917261 us; error 66 us; skew 500 ppm
Aug 11 22:15:18 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:18.920668 27448 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:15:18.921661 27448 webserver.cc:460] Webserver started at http://127.17.198.1:40273/ using document root <none> and password file <none>
I20260811 22:15:18.922282 27448 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:18.922391 27448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:18.922622 27448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:18.924973 27448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "206861f49a674b49a2659ac56289ca4b"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "0f317553216a79886970d7c4fa21c8ee"
server_key_iv: "b61ed1618a54cc9598bbdd30f03fc713"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.925609 27448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "206861f49a674b49a2659ac56289ca4b"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "0f317553216a79886970d7c4fa21c8ee"
server_key_iv: "b61ed1618a54cc9598bbdd30f03fc713"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.929788 27448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:18.932581 27465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.934260 27448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:18.934490 27448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "206861f49a674b49a2659ac56289ca4b"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "0f317553216a79886970d7c4fa21c8ee"
server_key_iv: "b61ed1618a54cc9598bbdd30f03fc713"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.934641 27448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:18.958254 27448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:18.961881 27448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:18.962131 27448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:18.962821 27448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:18.963760 27448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:18.963850 27448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.963925 27448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:18.963963 27448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.973663 27448 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:44183
I20260811 22:15:18.973686 27598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:44183 every 8 connection(s)
I20260811 22:15:18.974645 27448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:18.979242 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 27448
I20260811 22:15:18.979410 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:18.979692 18200 external_mini_cluster.cc:1428] Setting key 251b5f790b4053a2435afdeed00be2c4
Aug 11 22:15:18 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:18.990734 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.976689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:50233 (local address 127.17.198.62:39661)
0811 22:15:18.976902 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.976906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.978105 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 22:15:18.978988 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.978993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.979060 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.979171 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:18.980697 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.981494 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.982704 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.983128 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.983257 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.986169 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:18.986176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:18.986182 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:18.986221 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:18.988255 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.988870 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.988877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.988881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.989008 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.989379 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.989385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.989389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.989689 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:18.989801 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.990384 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.990535 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:15:18.991616 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.977181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:50233)
0811 22:15:18.977888 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.977962 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.978742 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.979333 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.979340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.979847 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:18.980539 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.980562 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.981636 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.981639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.982562 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.982572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.983298 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.983301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.983497 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.984182 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:18.984199 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:18.985972 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:18.988399 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.988405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.988420 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.988711 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.989115 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.989117 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.989118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.989272 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.989807 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:18.989813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:18.990115 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.990286 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.990576 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":135,"threads_started":1}
I20260811 22:15:18.993345 27599 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39661
I20260811 22:15:18.993630 27599 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:18.994230 27599 heartbeater.cc:511] Master 127.17.198.62:39661 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:15:18.995882 27372 ts_manager.cc:194] Registered new tserver with Master: 206861f49a674b49a2659ac56289ca4b (127.17.198.1:44183)
I20260811 22:15:18.997140 27372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:50233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:15:19.017493 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:19.139122 27605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:19.139428 27605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:19.139473 27605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:19.143781 27605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:19.143872 27605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:19.143936 27605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:19.150264 27605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:19.151584 27605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:19.152669 27605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:19.163477 27610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.163444 27611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:19.163548 27605 server_base.cc:1034] running on GCE node
W20260811 22:15:19.163444 27614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:19.164222 27605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:19.165105 27605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:19.166332 27605 hybrid_clock.cc:648] HybridClock initialized: now 1786486519166313 us; error 28 us; skew 500 ppm
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:19.169790 27605 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:15:19.170905 27605 webserver.cc:460] Webserver started at http://127.17.198.2:36455/ using document root <none> and password file <none>
I20260811 22:15:19.171504 27605 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:19.171574 27605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:19.171758 27605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:19.174158 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "e83b03c7c00346e99ce8dd9fe7b7885a"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "4154e6af0ef05e567fb8c3954d9833ba"
server_key_iv: "fa2259fe397bd5a31e7c9804b785ec44"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.174764 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "e83b03c7c00346e99ce8dd9fe7b7885a"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "4154e6af0ef05e567fb8c3954d9833ba"
server_key_iv: "fa2259fe397bd5a31e7c9804b785ec44"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.179276 27605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:19.182185 27622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.183723 27605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:19.183868 27605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "e83b03c7c00346e99ce8dd9fe7b7885a"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "4154e6af0ef05e567fb8c3954d9833ba"
server_key_iv: "fa2259fe397bd5a31e7c9804b785ec44"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.183988 27605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:19.199441 27605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:19.202826 27605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:19.203001 27605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:19.203606 27605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:19.204468 27605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:19.204520 27605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.204564 27605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:19.204581 27605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.214600 27605 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:40211
I20260811 22:15:19.214618 27753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:40211 every 8 connection(s)
I20260811 22:15:19.215600 27605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:19.216077 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 27605
I20260811 22:15:19.216213 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:19.216562 18200 external_mini_cluster.cc:1428] Setting key 6b7ecc8524da747c5592e9bf67b21990
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:15:19.234247 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.218082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:43735 (local address 127.17.198.62:39661)
0811 22:15:19.218267 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.218274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.219764 (+ 1490us) server_negotiation.cc:407] Connection header received
0811 22:15:19.220693 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.220696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.220755 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.220899 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:19.222394 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.223112 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.225209 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.225567 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.225719 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.229294 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:19.229299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:19.229302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:19.229337 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:19.231409 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.232211 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.232218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.232222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.232377 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.232857 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.232861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.232863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.233130 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:19.233260 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.233882 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.234050 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:15:19.235466 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.218684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:43735)
0811 22:15:19.219609 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.219659 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:19.220486 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:19.221087 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:19.221093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:19.221552 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:19.222178 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.222201 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.223318 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.223324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.225045 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.225061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.225788 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.225795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.226043 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.226773 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:19.226800 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:19.229046 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:19.231579 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:19.231585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:19.231603 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:19.232048 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:19.232506 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:19.232513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:19.232516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:19.232703 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:19.233260 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:19.233269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:19.233604 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 22:15:19.233829 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 22:15:19.234206 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":181,"threads_started":1}
I20260811 22:15:19.237061 27754 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39661
I20260811 22:15:19.237357 27754 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:19.238062 27754 heartbeater.cc:511] Master 127.17.198.62:39661 requested a full tablet report, sending...
I20260811 22:15:19.239432 27372 ts_manager.cc:194] Registered new tserver with Master: e83b03c7c00346e99ce8dd9fe7b7885a (127.17.198.2:40211)
I20260811 22:15:19.240170 27372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:43735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:15:19.253016 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:19.370584 27761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:19.370932 27761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:19.371039 27761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:19.375306 27761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:19.375422 27761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:19.375519 27761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:19.381487 27761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:19.382784 27761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:19.383890 27761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:19.394816 27769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.394919 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.394838 27768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:19.395076 27761 server_base.cc:1034] running on GCE node
I20260811 22:15:19.395642 27761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:19.396355 27761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:19.397569 27761 hybrid_clock.cc:648] HybridClock initialized: now 1786486519397537 us; error 49 us; skew 500 ppm
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:19.401155 27761 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:15:19.402195 27761 webserver.cc:460] Webserver started at http://127.17.198.3:46189/ using document root <none> and password file <none>
I20260811 22:15:19.402812 27761 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:19.402884 27761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:19.403079 27761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:19.405472 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "48f2a78b32cb4fe08a058ec7474ca422"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "903bf8e730b469c43147820f3a8b28d4"
server_key_iv: "2bdda8d06748757232fb8e268dad09a9"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.406159 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "48f2a78b32cb4fe08a058ec7474ca422"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "903bf8e730b469c43147820f3a8b28d4"
server_key_iv: "2bdda8d06748757232fb8e268dad09a9"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.410588 27761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:19.413563 27780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.415091 27761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:19.415283 27761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "48f2a78b32cb4fe08a058ec7474ca422"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "903bf8e730b469c43147820f3a8b28d4"
server_key_iv: "2bdda8d06748757232fb8e268dad09a9"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.415450 27761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:19.445075 27761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:19.449332 27761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:19.449628 27761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:19.450361 27761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:19.451440 27761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:19.451522 27761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.451598 27761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:19.451654 27761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.461992 27761 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:43843
I20260811 22:15:19.462040 27910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:43843 every 8 connection(s)
I20260811 22:15:19.463327 27761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:19.472281 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 27761
I20260811 22:15:19.472469 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:19.472895 18200 external_mini_cluster.cc:1428] Setting key ba11d2cd1a9e43ee1b6da82510a102fe
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:19.479269 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.465564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47073 (local address 127.17.198.62:39661)
0811 22:15:19.465737 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.465741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.466782 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 22:15:19.468099 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.468103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.468170 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.468261 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:19.469634 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.470217 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.471451 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.471675 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.471765 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.474970 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:19.474974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:19.474976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:19.475004 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:19.476669 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.477341 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.477344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.477345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.477449 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.477840 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.477843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.477844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.478068 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:19.478154 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.478980 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.479134 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:15:19.480213 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.466041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:47073)
0811 22:15:19.466609 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.466656 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:19.467840 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:19.468471 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:19.468478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:19.468913 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:19.469439 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.469456 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.470367 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.470370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.471316 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.471325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.471826 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.471829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.472093 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.472896 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:19.472921 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:19.474733 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:19.476796 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:19.476802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:19.476861 (+ 59us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:19.477172 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:19.477568 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:19.477571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:19.477572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:19.477733 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:19.478207 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:19.478214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:19.478570 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 22:15:19.478821 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:15:19.479175 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":142,"threads_started":1}
I20260811 22:15:19.481869 27911 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39661
I20260811 22:15:19.482204 27911 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:19.482829 27911 heartbeater.cc:511] Master 127.17.198.62:39661 requested a full tablet report, sending...
I20260811 22:15:19.484051 27372 ts_manager.cc:194] Registered new tserver with Master: 48f2a78b32cb4fe08a058ec7474ca422 (127.17.198.3:43843)
I20260811 22:15:19.484687 27372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:47073
I20260811 22:15:19.487762 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
Entry for principal oryx/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab.
WARNING: no policy specified for oryx/127.17.198.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
Entry for principal oryx/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab.
WARNING: no policy specified for oryx/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
Entry for principal oryx/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab.
I20260811 22:15:19.584596 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 27448
I20260811 22:15:19.592314 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 27605
I20260811 22:15:19.601305 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 27761
I20260811 22:15:19.611557 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 27338
I20260811 22:15:19.620728 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39661
--webserver_interface=127.17.198.62
--webserver_port=42661
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:39661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:19.738210 27923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:19.738569 27923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:19.738652 27923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:19.743074 27923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:19.743170 27923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:19.743202 27923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:19.743224 27923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:19.743253 27923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:19.750237 27923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:39661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=42661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:19.751361 27923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:19.752369 27923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:19.761744 27932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.761765 27930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.761790 27929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:19.762108 27923 server_base.cc:1034] running on GCE node
I20260811 22:15:19.762774 27923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:19.764050 27923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:19.765247 27923 hybrid_clock.cc:648] HybridClock initialized: now 1786486519765213 us; error 60 us; skew 500 ppm
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:19.768340 27923 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:15:19.769486 27923 webserver.cc:460] Webserver started at http://127.17.198.62:42661/ using document root <none> and password file <none>
I20260811 22:15:19.770123 27923 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:19.770210 27923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:19.774844 27923 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:19.777459 27939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.779055 27923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:19.779193 27923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "571ccd61cfad4a60aee27ca7333de0ae"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "5f008210a036f439e63f481bf1662d9a"
server_key_iv: "cc4f7962b247c3be74f1e6468d9f6618"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.779773 27923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:19.795431 27923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:19.799264 27923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:19.799448 27923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:19.807722 27923 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:39661
I20260811 22:15:19.807741 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:39661 every 8 connection(s)
I20260811 22:15:19.809003 27923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:19.813723 28000 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:15:19.815588 28000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Bootstrap starting.
I20260811 22:15:19.819161 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 27923
I20260811 22:15:19.819679 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:44183
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:39661
--webserver_port=40273
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:19.834686 28000 log.cc:824] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:19.842379 28000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:19.842859 28000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Bootstrap complete.
I20260811 22:15:19.847790 28000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:19.848320 28000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571ccd61cfad4a60aee27ca7333de0ae, State: Initialized, Role: FOLLOWER
I20260811 22:15:19.848950 28000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:19.849128 28000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:19.849237 28000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:19.849375 28000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:15:19.851733 28000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:19.852138 28000 leader_election.cc:302] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 571ccd61cfad4a60aee27ca7333de0ae; no voters:
I20260811 22:15:19.852519 28000 leader_election.cc:288] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:15:19.852715 28005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:15:19.853051 28005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [term 2 LEADER]: Becoming Leader. State: Replica: 571ccd61cfad4a60aee27ca7333de0ae, State: Running, Role: LEADER
I20260811 22:15:19.853440 28005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } }
I20260811 22:15:19.853979 28000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:19.855049 28007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "571ccd61cfad4a60aee27ca7333de0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } } }
I20260811 22:15:19.855218 28007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:19.855669 28008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 571ccd61cfad4a60aee27ca7333de0ae. Latest consensus state: current_term: 2 leader_uuid: "571ccd61cfad4a60aee27ca7333de0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571ccd61cfad4a60aee27ca7333de0ae" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39661 } } }
I20260811 22:15:19.855844 28008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:19.856221 28016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:19.859426 28016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:19.860095 28016 catalog_manager.cc:1259] Loaded cluster ID: 347e47092ca14067ba13141ce4ddfd79
I20260811 22:15:19.860193 28016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:19.862180 28016 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:19.862869 28016 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 571ccd61cfad4a60aee27ca7333de0ae: Loaded TSK: 0
I20260811 22:15:19.863924 28016 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:15:19.947996 28002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:19.948414 28002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:19.948508 28002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:19.952960 28002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:19.953102 28002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:19.953236 28002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:19.960075 28002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.1.keytab
--principal=oryx/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:44183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=40273
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:19.961380 28002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:19.962491 28002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:19.973182 28031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.973289 28033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:19.973196 28030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:19.974123 28002 server_base.cc:1034] running on GCE node
I20260811 22:15:19.974692 28002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:19.975459 28002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:19.976672 28002 hybrid_clock.cc:648] HybridClock initialized: now 1786486519976643 us; error 49 us; skew 500 ppm
Aug 11 22:15:19 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:19.980284 28002 init.cc:377] Logged in from keytab as oryx/127.17.198.1@KRBTEST.COM (short username oryx)
I20260811 22:15:19.981657 28002 webserver.cc:460] Webserver started at http://127.17.198.1:40273/ using document root <none> and password file <none>
I20260811 22:15:19.982319 28002 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:19.982453 28002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:19.987417 28002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:15:19.989934 28041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:19.991420 28002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:15:19.991582 28002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "206861f49a674b49a2659ac56289ca4b"
format_stamp: "Formatted at 2026-08-11 22:15:18 on dist-test-slave-77v9"
server_key: "0f317553216a79886970d7c4fa21c8ee"
server_key_iv: "b61ed1618a54cc9598bbdd30f03fc713"
server_key_version: "encryptionkey@0"
I20260811 22:15:19.992206 28002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:20.033234 28002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:20.036720 28002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:20.036991 28002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:20.037694 28002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:20.038671 28002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:20.038750 28002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.038825 28002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:20.038877 28002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.048878 28002 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:44183
I20260811 22:15:20.048933 28168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:44183 every 8 connection(s)
I20260811 22:15:20.050058 28002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:20.058531 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28002
I20260811 22:15:20.059041 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:40211
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:39661
--webserver_port=36455
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.065359 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.052635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:49871)
0811 22:15:20.053404 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.053472 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.054357 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.055715 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.055722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.056269 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.056800 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.056836 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.058277 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.058280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.059361 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.059372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.060546 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.060550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.060701 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.061447 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.061465 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.064365 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":166,"threads_started":1}
W20260811 22:15:20.065749 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.052508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:49871 (local address 127.17.198.62:39661)
0811 22:15:20.053113 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.053131 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.053607 (+ 476us) server_negotiation.cc:407] Connection header received
0811 22:15:20.054936 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.054957 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.055328 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.055618 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.057016 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.058112 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.059495 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.060331 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.060543 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.064729 (+ 4186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:49871: BlockingRecv error: recv got EOF from 127.17.198.1:49871 (error 108)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":120,"threads_started":1}
W20260811 22:15:20.066066 28169 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:39661 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.198.62:39661: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.073406 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.066685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:57423)
0811 22:15:20.066844 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.066864 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.066971 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.067284 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.067287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.067485 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.067596 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.067601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.068453 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.068459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.069147 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.069157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.069675 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.069679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.069780 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.070403 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.070416 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.073237 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:15:20.073720 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.066780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:57423 (local address 127.17.198.62:39661)
0811 22:15:20.066913 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.066917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.066933 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:20.067109 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.067112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.067168 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.067265 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.067727 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.068325 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.069289 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.069545 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.069672 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.073567 (+ 3895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:57423: BlockingRecv error: recv got EOF from 127.17.198.1:57423 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.088204 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.077130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:39791)
0811 22:15:20.077373 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.077408 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.077546 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.081730 (+ 4184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.081734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.081981 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.082118 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.082125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.083375 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.083378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.084285 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.084297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.085111 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.085114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.085208 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.085765 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.085786 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.088076 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 22:15:20.088573 28169 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:15:20.088884 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.081067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:39791 (local address 127.17.198.62:39661)
0811 22:15:20.081277 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.081283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.081307 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:15:20.081370 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.081374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.081438 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.081561 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.082293 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.083185 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.084438 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.084793 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.084906 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.088725 (+ 3819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:39791: BlockingRecv error: recv got EOF from 127.17.198.1:39791 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:15:20.188465 28173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:20.188939 28173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:20.189013 28173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:20.193975 28173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:20.194108 28173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:20.194204 28173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:20.201221 28173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.2.keytab
--principal=oryx/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:40211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=36455
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:20.202414 28173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:20.203506 28173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:20.214006 28182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.214202 28173 server_base.cc:1034] running on GCE node
W20260811 22:15:20.214000 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.214000 28180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.214833 28173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:20.215520 28173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:20.216714 28173 hybrid_clock.cc:648] HybridClock initialized: now 1786486520216680 us; error 46 us; skew 500 ppm
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:20.219813 28173 init.cc:377] Logged in from keytab as oryx/127.17.198.2@KRBTEST.COM (short username oryx)
I20260811 22:15:20.220849 28173 webserver.cc:460] Webserver started at http://127.17.198.2:36455/ using document root <none> and password file <none>
I20260811 22:15:20.221460 28173 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:20.221585 28173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:20.225987 28173 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:20.228430 28190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.230023 28173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:20.230188 28173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "e83b03c7c00346e99ce8dd9fe7b7885a"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "4154e6af0ef05e567fb8c3954d9833ba"
server_key_iv: "fa2259fe397bd5a31e7c9804b785ec44"
server_key_version: "encryptionkey@0"
I20260811 22:15:20.230819 28173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:20.245021 28173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:20.248728 28173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:20.248989 28173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:20.249643 28173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:20.250547 28173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:20.250623 28173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.250698 28173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:20.250741 28173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.261966 28173 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:40211
I20260811 22:15:20.261989 28319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:40211 every 8 connection(s)
I20260811 22:15:20.263111 28173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:20.268913 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28173
I20260811 22:15:20.269471 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:43843
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:39661
--webserver_port=46189
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:15:20.277333 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.265240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:34867 (local address 127.17.198.62:39661)
0811 22:15:20.265406 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.265411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.266728 (+ 1317us) server_negotiation.cc:407] Connection header received
0811 22:15:20.267733 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.267735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.267791 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.267876 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.269538 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.270375 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.271963 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.272253 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.272411 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.277181 (+ 4770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:34867: BlockingRecv error: recv got EOF from 127.17.198.2:34867 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:15:20.278326 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.265703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:34867)
0811 22:15:20.266525 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.266589 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.267516 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.268186 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.268194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.268627 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.269343 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.269361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.270574 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.270577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.271753 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.271767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.272469 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.272474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.272697 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.273560 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.273582 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.277008 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":140,"threads_started":1}
W20260811 22:15:20.278905 28320 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:39661 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.198.62:39661: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.285401 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.279435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:59623)
0811 22:15:20.279566 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.279585 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.279708 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.280075 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.280078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.280271 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.280402 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.280408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.281331 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.281335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.281895 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.281903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.282386 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.282390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.282482 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.283039 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.283051 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.285225 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:15:20.285498 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.279600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:59623 (local address 127.17.198.62:39661)
0811 22:15:20.279761 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.279767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.279786 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:20.279912 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.279914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.279957 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.280054 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.280540 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.281215 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.282032 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.282263 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.282379 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.285372 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:59623: BlockingRecv error: recv got EOF from 127.17.198.2:59623 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.291404 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.285995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:37241)
0811 22:15:20.286184 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.286198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.286281 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.286553 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.286556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.286729 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.286813 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.286818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.287574 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.287579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.288113 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.288121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.288593 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.288596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.288679 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.289262 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.289277 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.291236 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
W20260811 22:15:20.291504 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.286106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:37241 (local address 127.17.198.62:39661)
0811 22:15:20.286217 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.286223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.286238 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:20.286379 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.286382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.286423 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.286541 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.286940 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.287463 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.288238 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.288479 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.288592 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.291384 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:37241: BlockingRecv error: recv got EOF from 127.17.198.2:37241 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:15:20.291628 28320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:15:20.388659 28323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:20.389034 28323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:20.389114 28323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:20.393256 28323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:20.393332 28323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:20.393395 28323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:20.399184 28323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/krb5kdc/oryx_127.17.198.3.keytab
--principal=oryx/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:43843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=46189
--tserver_master_addrs=127.17.198.62:39661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:20.400354 28323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:20.401458 28323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:20.411862 28331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.411947 28332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.411918 28323 server_base.cc:1034] running on GCE node
W20260811 22:15:20.411859 28334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.412514 28323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:20.413295 28323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:20.414455 28323 hybrid_clock.cc:648] HybridClock initialized: now 1786486520414425 us; error 53 us; skew 500 ppm
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:20.417493 28323 init.cc:377] Logged in from keytab as oryx/127.17.198.3@KRBTEST.COM (short username oryx)
I20260811 22:15:20.418439 28323 webserver.cc:460] Webserver started at http://127.17.198.3:46189/ using document root <none> and password file <none>
I20260811 22:15:20.419118 28323 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:20.419241 28323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:20.423563 28323 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:20.425644 28342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.427105 28323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:20.427253 28323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "48f2a78b32cb4fe08a058ec7474ca422"
format_stamp: "Formatted at 2026-08-11 22:15:19 on dist-test-slave-77v9"
server_key: "903bf8e730b469c43147820f3a8b28d4"
server_key_iv: "2bdda8d06748757232fb8e268dad09a9"
server_key_version: "encryptionkey@0"
I20260811 22:15:20.427843 28323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:20.446058 28323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:20.449498 28323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:20.449728 28323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:20.450372 28323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:20.451262 28323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:20.451339 28323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.451416 28323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:20.451463 28323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.461938 28323 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:43843
I20260811 22:15:20.461946 28473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:43843 every 8 connection(s)
I20260811 22:15:20.463073 28323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:20.467849 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28323
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:15:20.475438 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.465124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47733 (local address 127.17.198.62:39661)
0811 22:15:20.465273 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.465276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.466317 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 22:15:20.467289 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.467292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.467353 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.467467 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.469058 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.469744 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.470921 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.471161 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.471280 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.475308 (+ 4028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:47733: BlockingRecv error: recv got EOF from 127.17.198.3:47733 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:15:20.476135 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.465492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:47733)
0811 22:15:20.466155 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.466198 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.467055 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.467692 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.467699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.468149 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.468895 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.468914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.469880 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.469883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.470784 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.470793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.471326 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.471330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.471519 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.472198 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.472226 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.475193 (+ 2967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":108,"threads_started":1}
W20260811 22:15:20.476785 28474 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:39661 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.198.62:39661: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
I20260811 22:15:20.478338 28480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.468979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.17.198.62:39661)
0811 22:15:20.469224 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.469228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.469240 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:15:20.469275 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.469277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.469318 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.469411 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.470163 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.470680 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.471555 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.471787 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.471900 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.473370 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:20.473378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:20.473401 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:20.473438 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:20.475354 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:20.475972 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:20.475978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:20.475983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:20.476109 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:20.476410 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:20.476418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:20.476419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:20.476982 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:20.477175 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:20.477869 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 22:15:20.478152 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.482956 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.477333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:41535)
0811 22:15:20.477561 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.477577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.477666 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.477943 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.477946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.478120 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.478236 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.478243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.479120 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.479125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.479721 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.479729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.480340 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.480344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.480436 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.481015 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.481027 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.482855 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 22:15:20.483179 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.477432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:41535 (local address 127.17.198.62:39661)
0811 22:15:20.477551 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.477554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.477620 (+ 66us) server_negotiation.cc:407] Connection header received
0811 22:15:20.477765 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.477771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.477824 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.477894 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.478373 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.478999 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.479882 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.480201 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.480341 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.483012 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:41535: BlockingRecv error: recv got EOF from 127.17.198.3:41535 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:20 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:20.488659 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.483596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:57473)
0811 22:15:20.483723 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.483741 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:20.483822 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:20.484077 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:20.484080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:20.484258 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:20.484332 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.484337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.485141 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.485145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.485661 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:20.485669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.486118 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.486122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:20.486200 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.486650 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:20.486661 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:20.488504 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:15:20.488729 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:20.483660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:57473 (local address 127.17.198.62:39661)
0811 22:15:20.483817 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:20.483821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:20.483834 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:15:20.483912 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:20.483915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:20.483976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:20.484061 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:20.484444 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.485034 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.485783 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:20.485998 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:20.486106 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:20.488622 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:57473: BlockingRecv error: recv got EOF from 127.17.198.3:57473 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 22:15:20.488945 28474 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:21.095875 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.089393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:38369)
0811 22:15:21.089702 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.089717 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:21.089824 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:21.090095 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:21.090097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:21.090286 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:21.090391 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.090396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.091299 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.091301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.091882 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.091892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.092409 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.092412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.092480 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.093189 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:21.093203 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:21.095577 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
W20260811 22:15:21.096079 28482 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:21.089457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38369 (local address 127.17.198.62:39661)
0811 22:15:21.089725 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:21.089729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:21.089874 (+ 145us) server_negotiation.cc:407] Connection header received
0811 22:15:21.089918 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:21.089920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:21.089968 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:21.090055 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:21.090515 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.091190 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.092017 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.092288 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.092384 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.095917 (+ 3533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:38369: BlockingRecv error: recv got EOF from 127.17.198.1:38369 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:21.298554 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.292392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:51961)
0811 22:15:21.292753 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.292771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:21.293019 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:21.293318 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:21.293321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:21.293549 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:21.293670 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.293677 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.294471 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.294476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.295071 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.295080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.295616 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.295619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.295752 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.296423 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:21.296438 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:21.298429 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":95,"threads_started":1}
W20260811 22:15:21.298766 28482 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:21.292542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:51961 (local address 127.17.198.62:39661)
0811 22:15:21.292738 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:21.292744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:21.292946 (+ 202us) server_negotiation.cc:407] Connection header received
0811 22:15:21.293123 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:21.293128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:21.293195 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:21.293291 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:21.293797 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.294351 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.295196 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.295436 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.295577 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.298617 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:51961: BlockingRecv error: recv got EOF from 127.17.198.2:51961 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:21 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:21.495796 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.489783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:37109)
0811 22:15:21.490104 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.490120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:21.490230 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:21.490523 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:21.490526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:21.490840 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:21.490965 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.490973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.491851 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.491854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.492478 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:21.492488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.493002 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.493005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:21.493073 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.493712 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:21.493724 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:21.495661 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":115,"threads_started":1}
W20260811 22:15:21.496008 28482 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:21.489797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:37109 (local address 127.17.198.62:39661)
0811 22:15:21.489943 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:21.489948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:21.490228 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:15:21.490304 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:21.490306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:21.490355 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:21.490435 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:21.491128 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.491680 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.492605 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:21.492857 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:21.492946 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:21.495858 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:37109: BlockingRecv error: recv got EOF from 127.17.198.3:37109 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:22.104024 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.097140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:55229)
0811 22:15:22.097468 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.097492 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:22.097598 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:22.097878 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:22.097880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:22.098075 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:22.098176 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.098182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.099158 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.099161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.099730 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.099739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.100265 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.100267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.100400 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.101232 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:22.101251 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:22.103771 (+ 2520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":120,"threads_started":1}
W20260811 22:15:22.104094 28487 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:22.097229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:55229 (local address 127.17.198.62:39661)
0811 22:15:22.097578 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:22.097582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:22.097601 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:22.097674 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:22.097677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:22.097721 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:22.097823 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:22.098341 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.099024 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.099889 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.100137 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.100237 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.103958 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:55229: BlockingRecv error: recv got EOF from 127.17.198.1:55229 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":151,"threads_started":1}
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:22.306562 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.299823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:39367)
0811 22:15:22.300220 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.300240 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:22.300370 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:22.300686 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:22.300688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:22.300939 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:22.301066 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.301073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.301907 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.301913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.302627 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.302638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.303189 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.303193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.303283 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.303897 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:22.303910 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:22.306324 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
W20260811 22:15:22.306631 28487 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:22.300085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:39367 (local address 127.17.198.62:39661)
0811 22:15:22.300291 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:22.300298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:22.300317 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:22.300502 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:22.300505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:22.300560 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:22.300663 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:22.301219 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.301781 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.302754 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.303064 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.303185 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.306509 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:39367: BlockingRecv error: recv got EOF from 127.17.198.2:39367 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:22 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:22.503688 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.496888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:45373)
0811 22:15:22.497230 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.497244 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:22.497354 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:22.497627 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:22.497629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:22.497824 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:22.497929 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.497934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.498760 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.498763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.499335 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.499344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.499956 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.499960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.500040 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.500706 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:22.500723 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:22.503475 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":153,"threads_started":1}
W20260811 22:15:22.503774 28487 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:22.497044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:45373 (local address 127.17.198.62:39661)
0811 22:15:22.497191 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:22.497195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:22.497318 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:15:22.497441 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:22.497443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:22.497492 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:22.497582 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:22.498089 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.498638 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.499466 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.499698 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.499801 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.503646 (+ 3845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:45373: BlockingRecv error: recv got EOF from 127.17.198.3:45373 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:23.112299 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.105162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:48249)
0811 22:15:23.105499 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.105513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:23.105623 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:23.105896 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:23.105898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:23.106140 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:23.106244 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.106250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.107276 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.107279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.107961 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.107973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.108600 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.108604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.108679 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.109488 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:23.109502 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:23.112142 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":101,"threads_started":1}
W20260811 22:15:23.112643 28491 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:23.105340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:48249 (local address 127.17.198.62:39661)
0811 22:15:23.105648 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:23.105652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:23.105668 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:23.105710 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:23.105712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:23.105753 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:23.105833 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:23.106407 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.107098 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.108144 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.108434 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.108556 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.112468 (+ 3912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:48249: BlockingRecv error: recv got EOF from 127.17.198.1:48249 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":136,"threads_started":1}
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:23.314496 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.307653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:57445)
0811 22:15:23.307988 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.308004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:23.308125 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:23.308529 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:23.308532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:23.308755 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:23.308918 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.308924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.309956 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.309960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.310603 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.310614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.311150 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.311152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.311221 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.311855 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:23.311868 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:23.314353 (+ 2485us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":108,"threads_started":1}
W20260811 22:15:23.314770 28491 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:23.307758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:57445 (local address 127.17.198.62:39661)
0811 22:15:23.307943 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:23.307946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:23.308140 (+ 194us) server_negotiation.cc:407] Connection header received
0811 22:15:23.308282 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:23.308285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:23.308344 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:23.308439 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:23.309094 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.309726 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.310781 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.311018 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.311119 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.314639 (+ 3520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:57445: BlockingRecv error: recv got EOF from 127.17.198.2:57445 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:23 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:23.515308 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.508104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:48049)
0811 22:15:23.508554 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.508568 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:23.508679 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:23.509060 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:23.509063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:23.509276 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:23.509385 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.509391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.510387 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.510391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.511007 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:23.511017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.511814 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.511818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:23.511914 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.512626 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:23.512639 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:23.515160 (+ 2521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":135,"threads_started":1}
W20260811 22:15:23.515534 28491 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:23.508295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:48049 (local address 127.17.198.62:39661)
0811 22:15:23.508485 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:23.508489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:23.508671 (+ 182us) server_negotiation.cc:407] Connection header received
0811 22:15:23.508755 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:23.508757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:23.508891 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:23.508975 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:23.509573 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.510232 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.511169 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:23.511647 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:23.511757 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:23.515392 (+ 3635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:48049: BlockingRecv error: recv got EOF from 127.17.198.3:48049 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:24.123615 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.113357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:55141)
0811 22:15:24.113744 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.113760 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:24.113864 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:24.114216 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:24.114218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:24.114434 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:24.114548 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.114554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.115394 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.115397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.115959 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.115967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.116420 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.116422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.116491 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.121087 (+ 4596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:24.121105 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:24.123395 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":98,"threads_started":1}
W20260811 22:15:24.123854 28496 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:24.113558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:55141 (local address 127.17.198.62:39661)
0811 22:15:24.113879 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:24.113883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:24.113898 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:24.114040 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:24.114043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:24.114090 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:24.114175 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:24.114677 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.115282 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.116072 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.116302 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.116386 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.123633 (+ 7247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:55141: BlockingRecv error: recv got EOF from 127.17.198.1:55141 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":130,"threads_started":1}
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:24.322216 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.315546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:50423)
0811 22:15:24.316022 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.316040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:24.316209 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:24.316514 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:24.316517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:24.316763 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:24.316895 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.316901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.317789 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.317793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.318370 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.318379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.318917 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.318921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.319017 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.319695 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:24.319709 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:24.322033 (+ 2324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":122,"threads_started":1}
W20260811 22:15:24.322346 28496 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:24.315596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:50423 (local address 127.17.198.62:39661)
0811 22:15:24.315985 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:24.315990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:24.316146 (+ 156us) server_negotiation.cc:407] Connection header received
0811 22:15:24.316325 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:24.316328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:24.316383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:24.316489 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:24.317038 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.317650 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.318514 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.318774 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.318893 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.322213 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:50423: BlockingRecv error: recv got EOF from 127.17.198.2:50423 (error 108)
Metrics: {"server-negotiator.queue_time_us":246}
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:24 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:24.523170 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.516470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:41025)
0811 22:15:24.516839 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.516859 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:24.516961 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:24.517277 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:24.517280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:24.517482 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:24.517586 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.517592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.518347 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.518349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.518978 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:24.518988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.519561 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.519564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:24.519647 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.520380 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:24.520393 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:24.523030 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":142,"threads_started":1}
W20260811 22:15:24.523447 28496 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:24.516621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:41025 (local address 127.17.198.62:39661)
0811 22:15:24.516779 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:24.516782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:24.516960 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:15:24.517037 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:24.517039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:24.517088 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:24.517176 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:24.517689 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.518234 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.519136 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:24.519372 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:24.519474 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:24.523261 (+ 3787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:41025: BlockingRecv error: recv got EOF from 127.17.198.3:41025 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:25.131407 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.124688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:45383)
0811 22:15:25.125088 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.125104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:25.125210 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:25.125518 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:25.125520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:25.125725 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:25.125822 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.125826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.126824 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.126827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.127415 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.127424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.128099 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.128102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.128184 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.128846 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:25.128865 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:25.131257 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":187,"threads_started":1}
W20260811 22:15:25.131703 28501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:25.124789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:45383 (local address 127.17.198.62:39661)
0811 22:15:25.125130 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:25.125135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:25.125162 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:15:25.125333 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:25.125335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:25.125382 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:25.125468 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:25.126045 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.126669 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.127568 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.127822 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.128013 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.131509 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:45383: BlockingRecv error: recv got EOF from 127.17.198.1:45383 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":82,"threads_started":1}
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:25.330140 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.323320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:54037)
0811 22:15:25.323790 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.323810 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:25.324001 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:25.324317 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:25.324319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:25.324579 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:25.324684 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.324689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.325732 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.325735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.326365 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.326375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.326981 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.326985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.327065 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.327707 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:25.327720 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:25.330006 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":222,"threads_started":1}
W20260811 22:15:25.330324 28501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:25.323465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:54037 (local address 127.17.198.62:39661)
0811 22:15:25.323649 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:25.323654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:25.323899 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:15:25.324123 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:25.324129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:25.324190 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:25.324274 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:25.324931 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.325598 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.326580 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.326846 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.326943 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.330189 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:54037: BlockingRecv error: recv got EOF from 127.17.198.2:54037 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:25 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:25.535501 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.524437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:59755)
0811 22:15:25.524983 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.525004 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:25.525148 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:25.525453 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:25.525456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:25.525712 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:25.525815 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.525821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.526664 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.526668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.527271 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:25.527279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.527833 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.527837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:25.527924 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.532241 (+ 4317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:25.532258 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:25.535310 (+ 3052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":139,"threads_started":1}
W20260811 22:15:25.535920 28501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:25.524614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:59755 (local address 127.17.198.62:39661)
0811 22:15:25.524876 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:25.524881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:25.525083 (+ 202us) server_negotiation.cc:407] Connection header received
0811 22:15:25.525274 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:25.525277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:25.525325 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:25.525420 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:25.525941 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.526538 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.527397 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:25.527645 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:25.527827 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:25.535636 (+ 7809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:59755: BlockingRecv error: recv got EOF from 127.17.198.3:59755 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:26.140213 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.132724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:34241)
0811 22:15:26.133134 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.133154 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:26.133311 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:26.133645 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:26.133648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:26.133959 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:26.134084 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.134116 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.135122 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.135125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.135881 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.135891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.136486 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.136489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.136570 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.137387 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:26.137410 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:26.140070 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":149,"threads_started":1}
W20260811 22:15:26.140590 28507 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:26.132888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:34241 (local address 127.17.198.62:39661)
0811 22:15:26.133244 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:26.133250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:26.133268 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:26.133436 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:26.133440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:26.133491 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:26.133619 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:26.134266 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.134967 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.136069 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.136320 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.136419 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.140433 (+ 4014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:34241: BlockingRecv error: recv got EOF from 127.17.198.1:34241 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:26.338199 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.331395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:48527)
0811 22:15:26.331684 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.331708 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:26.331812 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:26.332261 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:26.332264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:26.332508 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:26.332627 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.332633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.333617 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.333620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.334256 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.334266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.335007 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.335011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.335114 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.335875 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:26.335890 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:26.338070 (+ 2180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
W20260811 22:15:26.338408 28507 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:26.331522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:48527 (local address 127.17.198.62:39661)
0811 22:15:26.331836 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:26.331845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:26.331908 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:15:26.331959 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:26.331962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:26.332052 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:26.332138 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:26.332880 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.333491 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.334431 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.334783 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.335031 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.338259 (+ 3228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:48527: BlockingRecv error: recv got EOF from 127.17.198.2:48527 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:26 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:26.543756 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.536671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:42299)
0811 22:15:26.537283 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.537296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:26.537444 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:26.538006 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:26.538009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:26.538361 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:26.538538 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.538550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.539466 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.539472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.540129 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:26.540139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.540717 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.540722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:26.540897 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.541499 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:26.541515 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:26.543579 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":134,"threads_started":1}
W20260811 22:15:26.543884 28507 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:26.536991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:42299 (local address 127.17.198.62:39661)
0811 22:15:26.537181 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:26.537187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:26.537426 (+ 239us) server_negotiation.cc:407] Connection header received
0811 22:15:26.537639 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:26.537646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:26.537722 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:26.537975 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:26.538700 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.539332 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.540284 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:26.540559 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:26.540694 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:26.543747 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:42299: BlockingRecv error: recv got EOF from 127.17.198.3:42299 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:27.148198 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.141299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:44701)
0811 22:15:27.141687 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.141702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:27.141809 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:27.142194 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:27.142197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:27.142411 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:27.142518 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.142524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.143376 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.143380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.144005 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.144014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.144537 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.144540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.144609 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.145267 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:27.145280 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:27.148011 (+ 2731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":107,"threads_started":1}
W20260811 22:15:27.148295 28511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:27.141622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:44701 (local address 127.17.198.62:39661)
0811 22:15:27.141943 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:27.141947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:27.141963 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:27.142010 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:27.142012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:27.142056 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:27.142152 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:27.142661 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.143243 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.144152 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.144397 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.144504 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.148148 (+ 3644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:44701: BlockingRecv error: recv got EOF from 127.17.198.1:44701 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"spinlock_wait_cycles":8064,"thread_start_us":118,"threads_started":1}
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:27.345944 28512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.339223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:42897)
0811 22:15:27.339627 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.339642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:27.339747 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:27.340116 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:27.340118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:27.340347 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:27.340451 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.340457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.341441 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.341444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.342029 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.342038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.342586 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.342588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.342652 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.343311 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:27.343323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:27.345739 (+ 2416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":150,"threads_started":1}
W20260811 22:15:27.346123 28511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:27.339469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:42897 (local address 127.17.198.62:39661)
0811 22:15:27.339674 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:27.339687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:27.339705 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:27.339867 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:27.339871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:27.339925 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:27.340024 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:27.340601 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.341247 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.342176 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.342463 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.342563 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.345925 (+ 3362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:42897: BlockingRecv error: recv got EOF from 127.17.198.2:42897 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:27 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:27.552419 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.544998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:33281)
0811 22:15:27.545583 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.545603 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:27.545767 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:27.546109 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:27.546112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:27.546383 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:27.546504 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.546511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.547584 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.547588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.548285 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:27.548298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.549068 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.549072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:27.549149 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.549783 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:27.549795 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:27.552243 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":155,"threads_started":1}
W20260811 22:15:27.552632 28511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:27.545269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:33281 (local address 127.17.198.62:39661)
0811 22:15:27.545450 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:27.545455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:27.545761 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:15:27.545867 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:27.545870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:27.545932 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:27.546050 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:27.546652 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.547421 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.548476 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:27.548741 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:27.549036 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:27.552465 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:33281: BlockingRecv error: recv got EOF from 127.17.198.3:33281 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:28.156045 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.149129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:54911)
0811 22:15:28.149429 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.149443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:28.149541 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:28.149800 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:28.149802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:28.150024 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:28.150127 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.150133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.151119 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.151121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.151681 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.151689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.152298 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.152302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.152380 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.153183 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:28.153196 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:28.155855 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
W20260811 22:15:28.156141 28515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:28.149308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:54911 (local address 127.17.198.62:39661)
0811 22:15:28.149586 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:28.149590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:28.149602 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:15:28.149639 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:28.149641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:28.149681 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:28.149766 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:28.150285 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.150995 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.151875 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.152137 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.152236 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.156004 (+ 3768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:54911: BlockingRecv error: recv got EOF from 127.17.198.1:54911 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":116,"threads_started":1}
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:28.353578 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.346929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:46779)
0811 22:15:28.347346 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.347364 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:28.347528 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:28.347842 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:28.347845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:28.348119 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:28.348235 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.348241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.349344 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.349349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.349940 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.349949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.350454 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.350458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.350550 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.351138 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:28.351152 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:28.353443 (+ 2291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":90,"threads_started":1}
W20260811 22:15:28.353801 28515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:28.347067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:46779 (local address 127.17.198.62:39661)
0811 22:15:28.347231 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:28.347236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:28.347474 (+ 238us) server_negotiation.cc:407] Connection header received
0811 22:15:28.347657 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:28.347660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:28.347717 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:28.347814 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:28.348366 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.349225 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.350087 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.350323 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.350438 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.353659 (+ 3221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:46779: BlockingRecv error: recv got EOF from 127.17.198.2:46779 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:28 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:28.560549 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.553734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:44575)
0811 22:15:28.554120 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.554141 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:28.554311 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:28.554672 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:28.554676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:28.555002 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:28.555109 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.555115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.556079 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.556085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.556882 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:28.556894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.557585 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.557589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:28.557696 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.558347 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:28.558369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:28.560359 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":97,"threads_started":1}
W20260811 22:15:28.560624 28515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:28.553910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:44575 (local address 127.17.198.62:39661)
0811 22:15:28.554111 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:28.554116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:28.554245 (+ 129us) server_negotiation.cc:407] Connection header received
0811 22:15:28.554489 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:28.554491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:28.554545 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:28.554646 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:28.555242 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.555952 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.557031 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:28.557414 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:28.557585 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:28.560509 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:44575: BlockingRecv error: recv got EOF from 127.17.198.3:44575 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:29.168870 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.157296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:52971)
0811 22:15:29.157626 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.157649 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:29.157772 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:29.158166 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:29.158170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:29.158397 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:29.158499 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.158505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.159323 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.159329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.159896 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.159905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.160404 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.160408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.160493 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.166068 (+ 5575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:29.166086 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:29.168598 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
W20260811 22:15:29.169129 28519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:29.157439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:52971 (local address 127.17.198.62:39661)
0811 22:15:29.157722 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:29.157726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:29.157743 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:29.157961 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:29.157963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:29.158014 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:29.158135 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:29.158635 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.159201 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.160030 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.160272 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.160390 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.168806 (+ 8416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:52971: BlockingRecv error: recv got EOF from 127.17.198.1:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:29.361449 28520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.354632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:55697)
0811 22:15:29.355038 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.355057 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:29.355198 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:29.355576 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:29.355579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:29.355846 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:29.356019 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.356031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.357125 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.357128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.357719 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.357728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.358273 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.358276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.358360 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.358960 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:29.358977 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:29.361316 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":89,"threads_started":1}
W20260811 22:15:29.361804 28519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:29.354807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:55697 (local address 127.17.198.62:39661)
0811 22:15:29.355023 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:29.355026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:29.355151 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:15:29.355329 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:29.355332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:29.355397 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:29.355518 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:29.356140 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.356984 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.357853 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.358106 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.358198 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.361547 (+ 3349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:55697: BlockingRecv error: recv got EOF from 127.17.198.2:55697 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:29 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:29.569346 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.561830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:47497)
0811 22:15:29.562230 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.562246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:29.562375 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:29.562768 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:29.562770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:29.563040 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:29.563148 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.563154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.564320 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.564323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.565046 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:29.565068 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.565754 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.565758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:29.565850 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.566555 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:29.566568 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:29.569182 (+ 2614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":89,"threads_started":1}
W20260811 22:15:29.569571 28519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:29.562051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47497 (local address 127.17.198.62:39661)
0811 22:15:29.562295 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:29.562299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:29.562321 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:15:29.562530 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:29.562534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:29.562595 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:29.562685 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:29.563331 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.564111 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.565248 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:29.565581 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:29.565706 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:29.569403 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:47497: BlockingRecv error: recv got EOF from 127.17.198.3:47497 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:30.176577 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.170028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:59181)
0811 22:15:30.170429 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.170446 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:30.170556 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:30.170973 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:30.170976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:30.171176 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:30.171275 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.171280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.172104 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.172106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.172826 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.172873 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.173554 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.173564 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.173682 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.174300 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:30.174315 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:30.176397 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":109,"threads_started":1}
W20260811 22:15:30.176700 28524 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:30.170213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:59181 (local address 127.17.198.62:39661)
0811 22:15:30.170491 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:30.170495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:30.170512 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:30.170701 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:30.170704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:30.170766 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:30.170856 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:30.171401 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.171950 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.173020 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.173343 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.173464 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.176559 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:59181: BlockingRecv error: recv got EOF from 127.17.198.1:59181 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:30.368687 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.362395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:40029)
0811 22:15:30.362709 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.362725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:30.362888 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:30.363166 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:30.363168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:30.363397 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:30.363502 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.363508 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.364341 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.364344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.365127 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.365142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.365603 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.365606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.365676 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.366315 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:30.366330 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:30.368539 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":106,"threads_started":1}
W20260811 22:15:30.368934 28524 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:30.362564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:40029 (local address 127.17.198.62:39661)
0811 22:15:30.362749 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:30.362753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:30.362807 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:15:30.362996 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:30.362999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:30.363046 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:30.363117 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:30.363611 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.364199 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.365256 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.365486 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.365575 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.368748 (+ 3173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:40029: BlockingRecv error: recv got EOF from 127.17.198.2:40029 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:30 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:30.577224 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.570509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:58949)
0811 22:15:30.570915 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.570936 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:30.571075 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:30.571502 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:30.571507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:30.571828 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:30.571993 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.572003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.572970 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.572973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.573560 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:30.573570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.574146 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.574149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:30.574232 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.574888 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:30.574903 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:30.577085 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":93,"threads_started":1}
W20260811 22:15:30.577404 28524 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:30.570662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:58949 (local address 127.17.198.62:39661)
0811 22:15:30.570853 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:30.570858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:30.571064 (+ 206us) server_negotiation.cc:407] Connection header received
0811 22:15:30.571210 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:30.571213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:30.571320 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:30.571426 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:30.572147 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.572718 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.573709 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:30.574015 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:30.574109 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:30.577264 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:58949: BlockingRecv error: recv got EOF from 127.17.198.3:58949 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:31.183912 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.177501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:46083)
0811 22:15:31.177872 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.177895 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:31.178180 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:31.178560 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:31.178563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:31.178756 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:31.178858 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.178863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.179699 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.179701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.180261 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.180269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.180746 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.180749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.180882 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.181485 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:31.181497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:31.183717 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":92,"threads_started":1}
W20260811 22:15:31.184033 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:31.177594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:46083 (local address 127.17.198.62:39661)
0811 22:15:31.177987 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:31.177991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:31.178291 (+ 300us) server_negotiation.cc:407] Connection header received
0811 22:15:31.178340 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:31.178343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:31.178394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:31.178481 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:31.178976 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.179591 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.180392 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.180635 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.180734 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.183873 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:46083: BlockingRecv error: recv got EOF from 127.17.198.1:46083 (error 108)
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":228,"threads_started":1}
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:31.376185 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.369795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:56717)
0811 22:15:31.370129 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.370146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:31.370261 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:31.370540 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:31.370542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:31.370736 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:31.370836 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.370841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.371669 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.371672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.372287 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.372296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.372932 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.372939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.373025 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.373707 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:31.373722 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:31.376054 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":83,"threads_started":1}
W20260811 22:15:31.376389 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:31.369958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:56717 (local address 127.17.198.62:39661)
0811 22:15:31.370156 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:31.370160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:31.370225 (+ 65us) server_negotiation.cc:407] Connection header received
0811 22:15:31.370355 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:31.370358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:31.370412 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:31.370496 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:31.371009 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.371554 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.372451 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.372717 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.372859 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.376222 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:56717: BlockingRecv error: recv got EOF from 127.17.198.2:56717 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:31 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:31.585013 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.578267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:37609)
0811 22:15:31.578601 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.578616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:31.578717 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:31.578985 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:31.578987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:31.579270 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:31.579425 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.579433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.580257 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.580259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.580916 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:31.580926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.581425 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.581428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:31.581495 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.582195 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:31.582208 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:31.584872 (+ 2664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
W20260811 22:15:31.585245 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:31.578437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:37609 (local address 127.17.198.62:39661)
0811 22:15:31.578645 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:31.578649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:31.578671 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:15:31.578816 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:31.578818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:31.578867 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:31.578943 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:31.579570 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.580151 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.581056 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:31.581293 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:31.581380 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:31.585116 (+ 3736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:37609: BlockingRecv error: recv got EOF from 127.17.198.3:37609 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:32.191618 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.184891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:39693)
0811 22:15:32.185219 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.185241 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:32.185361 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:32.185704 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:32.185707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:32.185912 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:32.186027 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.186033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.186976 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.186980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.187685 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.187696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.188252 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.188255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.188316 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.189012 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:32.189024 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:32.191480 (+ 2456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":124,"threads_started":1}
W20260811 22:15:32.191939 28534 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:32.185059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:39693 (local address 127.17.198.62:39661)
0811 22:15:32.185329 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:32.185333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:32.185349 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:32.185460 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:32.185462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:32.185527 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:32.185623 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:32.186168 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.186853 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.187869 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.188135 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.188231 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.191776 (+ 3545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:39693: BlockingRecv error: recv got EOF from 127.17.198.1:39693 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:32.384570 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.377322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:34041)
0811 22:15:32.377662 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.377678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:32.377780 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:32.378156 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:32.378158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:32.378386 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:32.378501 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.378507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.379384 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.379387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.380039 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.380049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.380565 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.380568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.380907 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.381718 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:32.381833 (+ 115us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:32.384441 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":126,"threads_started":1}
W20260811 22:15:32.384781 28534 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:32.377446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:34041 (local address 127.17.198.62:39661)
0811 22:15:32.377699 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:32.377704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:32.377763 (+ 59us) server_negotiation.cc:407] Connection header received
0811 22:15:32.377886 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:32.377889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:32.377947 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:32.378068 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:32.378637 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.379230 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.380170 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.380404 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.380562 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.384636 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:34041: BlockingRecv error: recv got EOF from 127.17.198.2:34041 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:32 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:32.592777 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.585915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:37425)
0811 22:15:32.586221 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.586237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:32.586352 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:32.586647 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:32.586650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:32.586915 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:32.587033 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.587040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.587986 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.587990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.588841 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:32.588857 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.589399 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.589403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:32.589482 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.590254 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:32.590267 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:32.592616 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":112,"threads_started":1}
W20260811 22:15:32.593158 28534 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:32.586047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:37425 (local address 127.17.198.62:39661)
0811 22:15:32.586212 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:32.586216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:32.586361 (+ 145us) server_negotiation.cc:407] Connection header received
0811 22:15:32.586423 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:32.586426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:32.586491 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:32.586578 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:32.587192 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.587773 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.589011 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:32.589251 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:32.589380 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:32.592941 (+ 3561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:37425: BlockingRecv error: recv got EOF from 127.17.198.3:37425 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:33.198889 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.192690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:40581)
0811 22:15:33.192990 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.193006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:33.193103 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:33.193340 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:33.193342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:33.193524 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:33.193621 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.193626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.194469 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.194472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.195078 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.195087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.195640 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.195643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.195709 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.196278 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:33.196290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:33.198691 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":70,"threads_started":1}
W20260811 22:15:33.199215 28538 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:33.192776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:40581 (local address 127.17.198.62:39661)
0811 22:15:33.193086 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:33.193089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:33.193104 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:15:33.193173 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:33.193175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:33.193217 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:33.193296 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:33.193737 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.194333 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.195208 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.195544 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.195630 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.198983 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:40581: BlockingRecv error: recv got EOF from 127.17.198.1:40581 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:33.391755 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.385576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:34665)
0811 22:15:33.385976 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.385995 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:33.386160 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:33.386501 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:33.386504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:33.386742 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:33.386854 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.386860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.387681 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.387686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.388258 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.388266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.388739 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.388743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.388867 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.389527 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:33.389540 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:33.391558 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
W20260811 22:15:33.391829 28538 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:33.385784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:34665 (local address 127.17.198.62:39661)
0811 22:15:33.385976 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:33.385982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:33.386089 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:15:33.386291 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:33.386296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:33.386362 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:33.386471 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:33.386986 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.387549 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.388384 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.388617 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.388736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.391712 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:34665: BlockingRecv error: recv got EOF from 127.17.198.2:34665 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:33 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:33.600485 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.593786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:53811)
0811 22:15:33.594123 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.594136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:33.594276 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:33.594584 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:33.594586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:33.594809 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:33.594956 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.594962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.595828 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.595833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.596451 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:33.596463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.597082 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.597086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:33.597170 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.597858 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:33.597874 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:33.600293 (+ 2419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":122,"threads_started":1}
W20260811 22:15:33.600543 28538 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:33.593858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:53811 (local address 127.17.198.62:39661)
0811 22:15:33.594045 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:33.594051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:33.594218 (+ 167us) server_negotiation.cc:407] Connection header received
0811 22:15:33.594414 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:33.594417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:33.594465 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:33.594559 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:33.595089 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.595709 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.596653 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:33.596955 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:33.597074 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:33.600444 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:53811: BlockingRecv error: recv got EOF from 127.17.198.3:53811 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:34.211882 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.199924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:39629)
0811 22:15:34.200225 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.200262 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:34.200374 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:34.200951 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:34.200954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:34.201160 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:34.201272 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.201278 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.202317 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.202320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.202907 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.202916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.203446 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.203449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.203533 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.209509 (+ 5976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:34.209526 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:34.211730 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
W20260811 22:15:34.212289 28543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:34.200074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:39629 (local address 127.17.198.62:39661)
0811 22:15:34.200542 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:34.200546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:34.200562 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:15:34.200617 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:34.200619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:34.200667 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:34.200948 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:34.201437 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.202180 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.203057 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.203328 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.203428 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.212124 (+ 8696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:39629: BlockingRecv error: recv got EOF from 127.17.198.1:39629 (error 108)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":226,"threads_started":1}
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:34.399183 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.392626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:57333)
0811 22:15:34.393025 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.393040 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:34.393149 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:34.393428 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:34.393430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:34.393660 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:34.393908 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.393915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.394752 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.394758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.395446 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.395456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.396131 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.396135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.396214 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.396763 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:34.396776 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:34.399042 (+ 2266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":84,"threads_started":1}
W20260811 22:15:34.399374 28543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:34.392798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:57333 (local address 127.17.198.62:39661)
0811 22:15:34.393029 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:34.393032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:34.393111 (+ 79us) server_negotiation.cc:407] Connection header received
0811 22:15:34.393231 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:34.393233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:34.393279 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:34.393352 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:34.394036 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.394616 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.395593 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.395882 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.396100 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.399243 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:57333: BlockingRecv error: recv got EOF from 127.17.198.2:57333 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:34 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.3@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:34.607786 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.601408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.3:33407)
0811 22:15:34.601895 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.601909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:34.602078 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:34.602374 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:34.602377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:34.602616 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:34.602777 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.602782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.603617 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.603622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.604226 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:34.604236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.604743 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.604747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:34.604885 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.605516 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:34.605528 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:34.607572 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":104,"threads_started":1}
W20260811 22:15:34.607865 28543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:34.601587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:33407 (local address 127.17.198.62:39661)
0811 22:15:34.601804 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:34.601809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:34.602006 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:15:34.602188 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:34.602191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:34.602246 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:34.602352 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:34.602917 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.603495 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.604365 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:34.604613 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:34.604732 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:34.607730 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.3:33407: BlockingRecv error: recv got EOF from 127.17.198.3:33407 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 11 22:15:35 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:35 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.1@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:35.219214 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.213124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.1:55397)
0811 22:15:35.213416 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:35.213434 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:35.213558 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:35.213803 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:35.213806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:35.214034 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:35.214134 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:35.214140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.215060 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.215062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:35.215626 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:35.215634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.216157 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.216160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:35.216225 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:35.216825 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:35.216837 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:35.219086 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
W20260811 22:15:35.219499 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:35.213294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:55397 (local address 127.17.198.62:39661)
0811 22:15:35.213570 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:35.213574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:35.213597 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:15:35.213639 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:35.213641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:35.213685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:35.213771 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:35.214262 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.214928 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.215772 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.216027 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.216123 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:35.219288 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.1:55397: BlockingRecv error: recv got EOF from 127.17.198.1:55397 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":82,"threads_started":1}
Aug 11 22:15:35 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:15:35 dist-test-slave-77v9 krb5kdc[27323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.198.2@KRBTEST.COM for oryx/127.17.198.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:15:35.407171 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.400424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39661 (local address 127.17.198.2:47445)
0811 22:15:35.400703 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:35.400719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:35.400864 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:35.401162 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:35.401164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:35.401451 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:35.401602 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:35.401609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.402441 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.402444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:35.403124 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:35.403142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.403650 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.403655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:35.403738 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:35.404357 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:35.404373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:35.407001 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.198.62:39661: Server oryx/127.17.198.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":104,"threads_started":1}
W20260811 22:15:35.407387 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:35.400574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:47445 (local address 127.17.198.62:39661)
0811 22:15:35.400745 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:35.400750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:35.400770 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:15:35.400986 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:35.400988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:35.401038 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:35.401126 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:35.401726 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.402294 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.403277 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.403503 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.403592 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:35.407226 (+ 3634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.198.2:47445: BlockingRecv error: recv got EOF from 127.17.198.2:47445 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:15:35.478137 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28002
I20260811 22:15:35.492592 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28173
I20260811 22:15:35.501482 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28323
I20260811 22:15:35.512177 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 27923
2026-08-11T22:15:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18473 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:15:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:35Z Disabled control of system clock
I20260811 22:15:35.550158 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46883
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:15:35.673597 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:35.673952 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:35.673991 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:35.678352 28553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:35.678435 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:35.678452 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:35.678470 28553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:35.678486 28553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:35.683735 28553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:46883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:35.684979 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:35.686466 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:15:35.696513 28553 server_base.cc:1034] running on GCE node
W20260811 22:15:35.696519 28563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:35.696519 28559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:35.696542 28558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:35.697738 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:35.699144 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:35.700415 28553 hybrid_clock.cc:648] HybridClock initialized: now 1786486535700396 us; error 40 us; skew 500 ppm
I20260811 22:15:35.702992 28553 webserver.cc:460] Webserver started at http://127.17.198.62:45473/ using document root <none> and password file <none>
I20260811 22:15:35.703694 28553 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:35.703822 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:35.704097 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:35.706645 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "726886098bb74b63af63123df9ebbf6f"
format_stamp: "Formatted at 2026-08-11 22:15:35 on dist-test-slave-77v9"
server_key: "3b0709a1146ca72396b54d11c18be93d"
server_key_iv: "dc8ce6c651594fabde53e85e8d2301b8"
server_key_version: "encryptionkey@0"
I20260811 22:15:35.707376 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "726886098bb74b63af63123df9ebbf6f"
format_stamp: "Formatted at 2026-08-11 22:15:35 on dist-test-slave-77v9"
server_key: "3b0709a1146ca72396b54d11c18be93d"
server_key_iv: "dc8ce6c651594fabde53e85e8d2301b8"
server_key_version: "encryptionkey@0"
I20260811 22:15:35.711979 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:15:35.715415 28570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:35.716984 28553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:35.717192 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "726886098bb74b63af63123df9ebbf6f"
format_stamp: "Formatted at 2026-08-11 22:15:35 on dist-test-slave-77v9"
server_key: "3b0709a1146ca72396b54d11c18be93d"
server_key_iv: "dc8ce6c651594fabde53e85e8d2301b8"
server_key_version: "encryptionkey@0"
I20260811 22:15:35.717355 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:35.753748 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:35.754631 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:35.754873 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:35.762631 28553 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:46883
I20260811 22:15:35.762677 28626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:46883 every 8 connection(s)
I20260811 22:15:35.763767 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:35.767534 28627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:35.769238 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28553
I20260811 22:15:35.769387 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:35.769726 18200 external_mini_cluster.cc:1428] Setting key 112d238b3e468d09bc9f673beba1c317
I20260811 22:15:35.775502 28627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f: Bootstrap starting.
I20260811 22:15:35.778723 28627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:35.779848 28627 log.cc:824] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:35.782315 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.771431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.17.198.62:46883)
0811 22:15:35.772203 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:35.772226 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:35.772276 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:15:35.773323 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:35.773352 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:35.773952 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:35.774397 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:35.774877 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.776347 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.777487 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:35.778225 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:35.778405 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:35.779461 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:35.779468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:35.779487 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:35.780067 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:35.780149 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:35.780181 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:35.780297 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:35.780551 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 22:15:35.781140 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":119,"threads_started":1}
I20260811 22:15:35.783846 28627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f: No bootstrap required, opened a new log
I20260811 22:15:35.789880 28627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } }
I20260811 22:15:35.790234 28627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:35.790334 28627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726886098bb74b63af63123df9ebbf6f, State: Initialized, Role: FOLLOWER
I20260811 22:15:35.790901 28627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } }
I20260811 22:15:35.791123 28627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:35.791224 28627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:35.791348 28627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:35.792757 28627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } }
I20260811 22:15:35.793185 28627 leader_election.cc:302] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 726886098bb74b63af63123df9ebbf6f; no voters:
I20260811 22:15:35.793586 28627 leader_election.cc:288] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:35.793812 28634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:35.794152 28634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [term 1 LEADER]: Becoming Leader. State: Replica: 726886098bb74b63af63123df9ebbf6f, State: Running, Role: LEADER
I20260811 22:15:35.794524 28634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } }
I20260811 22:15:35.795126 28627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:35.796459 28636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "726886098bb74b63af63123df9ebbf6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } } }
I20260811 22:15:35.796618 28636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:35.797137 28647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:35.797078 28637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 726886098bb74b63af63123df9ebbf6f. Latest consensus state: current_term: 1 leader_uuid: "726886098bb74b63af63123df9ebbf6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726886098bb74b63af63123df9ebbf6f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46883 } } }
I20260811 22:15:35.797251 28637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:35.801404 28647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:35.808245 28647 catalog_manager.cc:1347] Generated new cluster ID: 89109b1fc85f44a794832aa74763f298
I20260811 22:15:35.808354 28647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:35.843230 28647 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:35.844244 28647 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:35.854501 28647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 726886098bb74b63af63123df9ebbf6f: Generated new TSK 0
I20260811 22:15:35.855441 28647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:35.870966 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--builtin_ntp_servers=127.17.198.20:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:15:35.994390 28660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:35.994881 28660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:35.994927 28660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:35.999660 28660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:35.999738 28660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:35.999797 28660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:36.005211 28660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:36.006601 28660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:36.008214 28660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:36.020215 28672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.020196 28660 server_base.cc:1034] running on GCE node
W20260811 22:15:36.020164 28667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:36.020217 28670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.021035 28660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:36.021875 28660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:36.023113 28660 hybrid_clock.cc:648] HybridClock initialized: now 1786486536023096 us; error 35 us; skew 500 ppm
I20260811 22:15:36.025787 28660 webserver.cc:460] Webserver started at http://127.17.198.1:38815/ using document root <none> and password file <none>
I20260811 22:15:36.026469 28660 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:36.026607 28660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:36.026877 28660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:36.029325 28660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "1bd10820585f409c92f39cd11d4e6514"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "25fbc0166a79b8391ed227570b7bf7b8"
server_key_iv: "ca2459ffd1d283abacc17e0307ebe0e5"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.030005 28660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "1bd10820585f409c92f39cd11d4e6514"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "25fbc0166a79b8391ed227570b7bf7b8"
server_key_iv: "ca2459ffd1d283abacc17e0307ebe0e5"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.034399 28660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:36.037851 28681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.039376 28660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:15:36.039595 28660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "1bd10820585f409c92f39cd11d4e6514"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "25fbc0166a79b8391ed227570b7bf7b8"
server_key_iv: "ca2459ffd1d283abacc17e0307ebe0e5"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.039762 28660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:36.057503 28660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:36.058461 28660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:36.058701 28660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:36.059363 28660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:36.060453 28660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:36.060531 28660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.060601 28660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:36.060652 28660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.070724 28660 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:37263
I20260811 22:15:36.070751 28821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:37263 every 8 connection(s)
I20260811 22:15:36.071859 28660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:36.080215 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28660
I20260811 22:15:36.080399 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:36.080758 18200 external_mini_cluster.cc:1428] Setting key 0fd1ea3c4053921334f80d7d2151dd92
I20260811 22:15:36.083896 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--builtin_ntp_servers=127.17.198.20:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:15:36.084790 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.074390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:56129 (local address 127.17.198.62:46883)
0811 22:15:36.074566 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.074570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.075957 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 22:15:36.077669 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.077674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.077764 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.077894 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.079066 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.079673 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.081785 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.082113 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.082221 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.084042 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.084050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.084055 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:36.084074 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.084105 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:36.084120 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.084215 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.084457 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.084560 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:15:36.085701 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.074849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46883 (local address 127.17.198.1:56129)
0811 22:15:36.075712 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.075781 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:36.077383 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:36.078163 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:36.078171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:36.078280 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:36.078877 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.078892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.079905 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.079909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.081536 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.081558 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.082332 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.082337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.082596 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.083668 (+ 1072us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:36.083669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:36.083725 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:36.083821 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:36.084270 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:36.084281 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:36.084287 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:15:36.084424 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:15:36.084718 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":142,"threads_started":1}
I20260811 22:15:36.087342 28822 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46883
I20260811 22:15:36.087644 28822 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:36.088287 28822 heartbeater.cc:511] Master 127.17.198.62:46883 requested a full tablet report, sending...
I20260811 22:15:36.090016 28590 ts_manager.cc:194] Registered new tserver with Master: 1bd10820585f409c92f39cd11d4e6514 (127.17.198.1:37263)
I20260811 22:15:36.091318 28590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.1:56129
W20260811 22:15:36.204730 28825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:36.205097 28825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:36.205173 28825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:36.209486 28825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:36.209568 28825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:36.209628 28825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:36.214732 28825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:36.215981 28825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:36.217433 28825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:36.228245 28832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:36.228199 28831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.228312 28825 server_base.cc:1034] running on GCE node
W20260811 22:15:36.228186 28834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.228998 28825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:36.229739 28825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:36.230962 28825 hybrid_clock.cc:648] HybridClock initialized: now 1786486536230928 us; error 48 us; skew 500 ppm
I20260811 22:15:36.233398 28825 webserver.cc:460] Webserver started at http://127.17.198.2:46825/ using document root <none> and password file <none>
I20260811 22:15:36.234059 28825 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:36.234164 28825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:36.234418 28825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:36.236734 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "885ca7a05da842e9986da4c277096287"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "d58810173b0316cc728f12b42077a870"
server_key_iv: "d1e5c1ec75412e5d77e050baaf4d8658"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.237447 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "885ca7a05da842e9986da4c277096287"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "d58810173b0316cc728f12b42077a870"
server_key_iv: "d1e5c1ec75412e5d77e050baaf4d8658"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.241603 28825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:36.244925 28845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.246460 28825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:36.246666 28825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "885ca7a05da842e9986da4c277096287"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "d58810173b0316cc728f12b42077a870"
server_key_iv: "d1e5c1ec75412e5d77e050baaf4d8658"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.246825 28825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:36.261346 28825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:36.262126 28825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:36.262357 28825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:36.263002 28825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:36.264055 28825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:36.264132 28825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.264210 28825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:36.264263 28825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.274433 28825 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:37583
I20260811 22:15:36.274449 28981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:37583 every 8 connection(s)
I20260811 22:15:36.275523 28825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:36.283262 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28825
I20260811 22:15:36.283422 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:36.283691 18200 external_mini_cluster.cc:1428] Setting key ffa23a3d11293ce658a5389e0a5d825a
I20260811 22:15:36.286217 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--builtin_ntp_servers=127.17.198.20:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:15:36.286495 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.277569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:45331 (local address 127.17.198.62:46883)
0811 22:15:36.277737 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.277743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.279017 (+ 1274us) server_negotiation.cc:407] Connection header received
0811 22:15:36.280484 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.280491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.280548 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.280642 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.281730 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.282490 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.284007 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.284289 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.284448 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.285888 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.285892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.285895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:36.285910 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.285935 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:36.285948 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.286034 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.286195 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.286275 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:15:36.287564 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.278090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46883 (local address 127.17.198.2:45331)
0811 22:15:36.278837 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.278888 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:36.280224 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:36.280878 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:36.280894 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:36.281021 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:36.281565 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.281578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.282683 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.282687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.283868 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.283880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.284501 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.284504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.284727 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.285589 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:36.285590 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:36.285637 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:36.285709 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:36.286044 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:36.286051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:36.286056 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:15:36.286163 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:15:36.286534 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":144,"threads_started":1}
I20260811 22:15:36.288889 28982 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46883
I20260811 22:15:36.289147 28982 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:36.289712 28982 heartbeater.cc:511] Master 127.17.198.62:46883 requested a full tablet report, sending...
I20260811 22:15:36.290838 28590 ts_manager.cc:194] Registered new tserver with Master: 885ca7a05da842e9986da4c277096287 (127.17.198.2:37583)
I20260811 22:15:36.291463 28590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.2:45331
W20260811 22:15:36.409897 28986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:36.410279 28986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:36.410362 28986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:36.415129 28986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:36.415258 28986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:36.415376 28986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:36.421347 28986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:36.422781 28986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:36.424348 28986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:36.435817 28995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.435969 28986 server_base.cc:1034] running on GCE node
W20260811 22:15:36.435813 28992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:36.435813 28993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.436569 28986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:36.437394 28986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:36.438592 28986 hybrid_clock.cc:648] HybridClock initialized: now 1786486536438547 us; error 62 us; skew 500 ppm
I20260811 22:15:36.441143 28986 webserver.cc:460] Webserver started at http://127.17.198.3:42251/ using document root <none> and password file <none>
I20260811 22:15:36.441867 28986 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:36.441949 28986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:36.442219 28986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:36.444986 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "32b53a99d04249c9a48651ec3fe3dfe9"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "f6392ce54578410ad6fc895151f57291"
server_key_iv: "f45bf0b8d52cb8ccde00e79abeb8d3d7"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.445724 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "32b53a99d04249c9a48651ec3fe3dfe9"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "f6392ce54578410ad6fc895151f57291"
server_key_iv: "f45bf0b8d52cb8ccde00e79abeb8d3d7"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.450290 28986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:15:36.453558 29003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.455217 28986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:36.455436 28986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "32b53a99d04249c9a48651ec3fe3dfe9"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "f6392ce54578410ad6fc895151f57291"
server_key_iv: "f45bf0b8d52cb8ccde00e79abeb8d3d7"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.455611 28986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:36.471092 28986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:36.471926 28986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:36.472172 28986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:36.472930 28986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:36.474012 28986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:36.474090 28986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.474169 28986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:36.474200 28986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.484939 28986 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:43785
I20260811 22:15:36.485009 29130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:43785 every 8 connection(s)
I20260811 22:15:36.486006 28986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:36.494931 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 28986
I20260811 22:15:36.495061 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:36.495496 18200 external_mini_cluster.cc:1428] Setting key dc1306cf6f526b20fcd6a37b7bdf58bb
I20260811 22:15:36.498209 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.488747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:42957 (local address 127.17.198.62:46883)
0811 22:15:36.489122 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.489126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.490368 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 22:15:36.492010 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.492013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.492067 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.492154 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.493369 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.494012 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.495425 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.495762 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.495905 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.497625 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.497629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.497632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:36.497646 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.497676 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:36.497685 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.497780 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.497989 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.498064 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272}
I20260811 22:15:36.499181 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.489303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46883 (local address 127.17.198.3:42957)
0811 22:15:36.490178 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.490228 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:36.491737 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:36.492423 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:36.492431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:36.492524 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:36.493175 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.493192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.494166 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.494169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.495280 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:36.495291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.496030 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.496036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:36.496386 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.497256 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:36.497258 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:36.497306 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:36.497436 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:36.497785 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:36.497798 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:36.497805 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:15:36.497969 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:15:36.498307 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":160,"threads_started":1}
I20260811 22:15:36.500434 29131 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46883
I20260811 22:15:36.500713 29131 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:36.501343 29131 heartbeater.cc:511] Master 127.17.198.62:46883 requested a full tablet report, sending...
I20260811 22:15:36.502542 28590 ts_manager.cc:194] Registered new tserver with Master: 32b53a99d04249c9a48651ec3fe3dfe9 (127.17.198.3:43785)
I20260811 22:15:36.503185 28590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.3:42957
I20260811 22:15:36.510720 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:15:36.514165 28632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:36.513005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.17.198.62:46883)
0811 22:15:36.513195 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.513201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.513358 (+ 157us) server_negotiation.cc:407] Connection header received
0811 22:15:36.513563 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.513566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.513613 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.513706 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.514011 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52674: BlockingRecv error: recv got EOF from 127.0.0.1:52674 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:15:36.514386 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28660
I20260811 22:15:36.524451 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28825
I20260811 22:15:36.532222 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28986
I20260811 22:15:36.539119 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 28553
2026-08-11T22:15:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1026 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:36Z Disabled control of system clock
I20260811 22:15:36.576508 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:38365
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:38365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:15:36.702733 29143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:36.703059 29143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:36.703099 29143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:36.707805 29143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:36.707898 29143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:36.707919 29143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:36.707937 29143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:36.707953 29143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:36.713135 29143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:38365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:38365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:36.714442 29143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:36.715891 29143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:36.725620 29150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:36.725620 29153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:36.725615 29151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:36.726001 29143 server_base.cc:1034] running on GCE node
I20260811 22:15:36.726684 29143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:36.727969 29143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:36.729176 29143 hybrid_clock.cc:648] HybridClock initialized: now 1786486536729144 us; error 49 us; skew 500 ppm
I20260811 22:15:36.731348 29143 webserver.cc:460] Webserver started at http://127.17.198.62:38025/ using document root <none> and password file <none>
I20260811 22:15:36.731993 29143 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:36.732095 29143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:36.732349 29143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:36.734661 29143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "2ff95cc873de445d8317c5565c8a86e4"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "fea155d531af0c1832cb68196d1eee74"
server_key_iv: "e30df416a11a5068b67045bfef40a443"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.735332 29143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "2ff95cc873de445d8317c5565c8a86e4"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "fea155d531af0c1832cb68196d1eee74"
server_key_iv: "e30df416a11a5068b67045bfef40a443"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.739895 29143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:15:36.743160 29162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:36.744709 29143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:15:36.744946 29143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "2ff95cc873de445d8317c5565c8a86e4"
format_stamp: "Formatted at 2026-08-11 22:15:36 on dist-test-slave-77v9"
server_key: "fea155d531af0c1832cb68196d1eee74"
server_key_iv: "e30df416a11a5068b67045bfef40a443"
server_key_version: "encryptionkey@0"
I20260811 22:15:36.745112 29143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:36.750468 29143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:36.751168 29143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:36.751385 29143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:36.759131 29143 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:38365
I20260811 22:15:36.759146 29222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:38365 every 8 connection(s)
I20260811 22:15:36.760177 29143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:36.763724 29223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:36.764669 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29143
I20260811 22:15:36.764829 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:36.765183 18200 external_mini_cluster.cc:1428] Setting key d48b7fff1b85263218e142334734c45e
I20260811 22:15:36.770960 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4: Bootstrap starting.
I20260811 22:15:36.771610 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.766845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.17.198.62:38365)
0811 22:15:36.767558 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.767576 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.767614 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:15:36.768292 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.768310 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.768712 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.769159 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.770126 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.770131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.770153 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:36.770165 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.770224 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:36.770240 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.770362 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.770509 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.770805 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":168,"threads_started":1}
I20260811 22:15:36.773547 29223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:36.774495 29223 log.cc:824] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:36.776723 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4: No bootstrap required, opened a new log
I20260811 22:15:36.780999 29223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } }
I20260811 22:15:36.781301 29223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:36.781339 29223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff95cc873de445d8317c5565c8a86e4, State: Initialized, Role: FOLLOWER
I20260811 22:15:36.781744 29223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } }
I20260811 22:15:36.781929 29223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:36.781997 29223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:36.782112 29223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:36.783442 29223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } }
I20260811 22:15:36.783823 29223 leader_election.cc:302] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ff95cc873de445d8317c5565c8a86e4; no voters:
I20260811 22:15:36.784140 29223 leader_election.cc:288] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:36.784382 29231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:36.784672 29231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff95cc873de445d8317c5565c8a86e4, State: Running, Role: LEADER
I20260811 22:15:36.785079 29231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } }
I20260811 22:15:36.785804 29223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:36.787015 29232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ff95cc873de445d8317c5565c8a86e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } } }
I20260811 22:15:36.787187 29232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:36.787514 29233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ff95cc873de445d8317c5565c8a86e4. Latest consensus state: current_term: 1 leader_uuid: "2ff95cc873de445d8317c5565c8a86e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff95cc873de445d8317c5565c8a86e4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38365 } } }
I20260811 22:15:36.787633 29240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:36.787706 29233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:36.791071 29240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:36.797919 29240 catalog_manager.cc:1347] Generated new cluster ID: 57730337890f4fdb810aeb011bcd4454
I20260811 22:15:36.798055 29240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:36.832741 29240 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:36.833190 29240 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:36.860270 29240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ff95cc873de445d8317c5565c8a86e4: Generated new TSK 0
I20260811 22:15:36.861366 29240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:36.880375 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--builtin_ntp_servers=127.17.198.20:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:15:37.007082 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:37.007403 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:37.007447 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:37.011855 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:37.011950 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:37.012010 29250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:37.017223 29250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:37.018529 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:37.020187 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:37.031811 29257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:37.031929 29259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:37.032011 29262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:37.032245 29250 server_base.cc:1034] running on GCE node
I20260811 22:15:37.032764 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:37.033602 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:37.034835 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786486537034800 us; error 49 us; skew 500 ppm
I20260811 22:15:37.037303 29250 webserver.cc:460] Webserver started at http://127.17.198.1:35929/ using document root <none> and password file <none>
I20260811 22:15:37.038301 29250 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:37.038400 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:37.038653 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:37.041124 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "68ce75b7d5224a43a5c57f8fd1d7f65e"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "bc8cb2c3fe4ba6fcc96235d6ea0c6967"
server_key_iv: "1639bd9b2e17c108c48d599b128dbd92"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.041816 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "68ce75b7d5224a43a5c57f8fd1d7f65e"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "bc8cb2c3fe4ba6fcc96235d6ea0c6967"
server_key_iv: "1639bd9b2e17c108c48d599b128dbd92"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.046077 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:15:37.049165 29270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.050843 29250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:37.051036 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "68ce75b7d5224a43a5c57f8fd1d7f65e"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "bc8cb2c3fe4ba6fcc96235d6ea0c6967"
server_key_iv: "1639bd9b2e17c108c48d599b128dbd92"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.051192 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:37.056627 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:37.057472 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:37.057703 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:37.058357 29250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:37.059428 29250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:37.059506 29250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.059584 29250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:37.059638 29250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.070353 29250 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:34751
I20260811 22:15:37.070396 29402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:34751 every 8 connection(s)
I20260811 22:15:37.071426 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:37.078898 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.073906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:54425 (local address 127.17.198.62:38365)
0811 22:15:37.074093 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:37.074098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:37.075240 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 22:15:37.076765 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:37.076769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:37.076840 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:37.076942 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:37.078269 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:37.078273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:37.078278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:37.078306 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:37.078329 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:37.078339 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:37.078494 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:37.078683 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:15:37.078740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:15:37.080080 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.074169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38365 (local address 127.17.198.1:54425)
0811 22:15:37.075049 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:37.075102 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:37.076505 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:37.077156 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:37.077167 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:37.077275 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:37.077934 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:37.077937 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:37.077993 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:37.078072 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:37.078508 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:37.078521 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:37.078528 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:15:37.078735 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:15:37.079032 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":140,"threads_started":1}
I20260811 22:15:37.080175 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29250
I20260811 22:15:37.080286 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:37.080684 18200 external_mini_cluster.cc:1428] Setting key 96a698e9d4618cd6e3481ffcc026434d
I20260811 22:15:37.081938 29403 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38365
I20260811 22:15:37.082219 29403 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:37.082710 29403 heartbeater.cc:511] Master 127.17.198.62:38365 requested a full tablet report, sending...
I20260811 22:15:37.083280 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--builtin_ntp_servers=127.17.198.20:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:15:37.084475 29179 ts_manager.cc:194] Registered new tserver with Master: 68ce75b7d5224a43a5c57f8fd1d7f65e (127.17.198.1:34751)
W20260811 22:15:37.207618 29406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:37.207944 29406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:37.207988 29406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:37.212493 29406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:37.212574 29406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:37.212637 29406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:37.217886 29406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:37.219138 29406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:37.220705 29406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:37.231227 29416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:37.231341 29413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:37.231227 29412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:37.231555 29406 server_base.cc:1034] running on GCE node
I20260811 22:15:37.232178 29406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:37.233011 29406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:37.234247 29406 hybrid_clock.cc:648] HybridClock initialized: now 1786486537234222 us; error 45 us; skew 500 ppm
I20260811 22:15:37.237035 29406 webserver.cc:460] Webserver started at http://127.17.198.2:34815/ using document root <none> and password file <none>
I20260811 22:15:37.237704 29406 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:37.237834 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:37.238103 29406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:37.240480 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "950069b5203d4ef883ce3c178839ac47"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "c8bab217dbe646b5cfd4a5bcb80b41c2"
server_key_iv: "468ba3db5e25ae6f07c06f063080d7cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.241220 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "950069b5203d4ef883ce3c178839ac47"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "c8bab217dbe646b5cfd4a5bcb80b41c2"
server_key_iv: "468ba3db5e25ae6f07c06f063080d7cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.246127 29406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:37.249567 29422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.251165 29406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:37.251351 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "950069b5203d4ef883ce3c178839ac47"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "c8bab217dbe646b5cfd4a5bcb80b41c2"
server_key_iv: "468ba3db5e25ae6f07c06f063080d7cd"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.251506 29406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:37.256796 29406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:37.257550 29406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:37.257759 29406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:37.258363 29406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:37.259387 29406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:37.259461 29406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.259554 29406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:37.259595 29406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.269387 29406 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:36713
I20260811 22:15:37.269423 29557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:36713 every 8 connection(s)
I20260811 22:15:37.270447 29406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:37.272770 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29406
I20260811 22:15:37.272985 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:37.273289 18200 external_mini_cluster.cc:1428] Setting key e290983df1cc6c9fe5fe8f9692216be8
I20260811 22:15:37.275575 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--builtin_ntp_servers=127.17.198.20:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:15:37.277505 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.272526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:51045 (local address 127.17.198.62:38365)
0811 22:15:37.272723 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:37.272728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:37.273978 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 22:15:37.275351 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:37.275357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:37.275413 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:37.275520 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:37.276881 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:37.276887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:37.276894 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:37.276912 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:37.276945 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:37.276959 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:37.277096 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:37.277241 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 22:15:37.277294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:15:37.278674 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.272998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38365 (local address 127.17.198.2:51045)
0811 22:15:37.273804 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:37.273860 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:37.275119 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:37.275736 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:37.275748 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:37.275874 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:37.276583 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:37.276585 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:37.276627 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:37.276699 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:37.277090 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:37.277098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:37.277103 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:15:37.277250 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:15:37.277547 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":149,"threads_started":1}
I20260811 22:15:37.280026 29558 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38365
I20260811 22:15:37.280301 29558 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:37.280776 29558 heartbeater.cc:511] Master 127.17.198.62:38365 requested a full tablet report, sending...
I20260811 22:15:37.282191 29179 ts_manager.cc:194] Registered new tserver with Master: 950069b5203d4ef883ce3c178839ac47 (127.17.198.2:36713)
W20260811 22:15:37.399654 29561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:37.399982 29561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:37.400030 29561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:37.404347 29561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:37.404430 29561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:37.404492 29561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:37.409871 29561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:37.411051 29561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:37.412494 29561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:37.423017 29569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:37.423017 29573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:37.423159 29561 server_base.cc:1034] running on GCE node
W20260811 22:15:37.423374 29570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:37.423832 29561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:37.424595 29561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:37.425781 29561 hybrid_clock.cc:648] HybridClock initialized: now 1786486537425763 us; error 80 us; skew 500 ppm
I20260811 22:15:37.428476 29561 webserver.cc:460] Webserver started at http://127.17.198.3:33715/ using document root <none> and password file <none>
I20260811 22:15:37.429491 29561 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:37.429601 29561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:37.429909 29561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:37.433501 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "84cbf66969744a12aec10da2f8ead9ad"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "e16971c4ac02275e91571fabb14e401a"
server_key_iv: "8503e66199a58e5a1029a36f6dd933e0"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.434454 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "84cbf66969744a12aec10da2f8ead9ad"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "e16971c4ac02275e91571fabb14e401a"
server_key_iv: "8503e66199a58e5a1029a36f6dd933e0"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.439116 29561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:15:37.442550 29585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.444376 29561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:37.444566 29561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "84cbf66969744a12aec10da2f8ead9ad"
format_stamp: "Formatted at 2026-08-11 22:15:37 on dist-test-slave-77v9"
server_key: "e16971c4ac02275e91571fabb14e401a"
server_key_iv: "8503e66199a58e5a1029a36f6dd933e0"
server_key_version: "encryptionkey@0"
I20260811 22:15:37.444751 29561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:37.450528 29561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:37.451243 29561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:37.451429 29561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:37.452056 29561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:37.453217 29561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:37.453272 29561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.453320 29561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:37.453351 29561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:37.464304 29561 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:40793
I20260811 22:15:37.464362 29730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:40793 every 8 connection(s)
I20260811 22:15:37.465713 29561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:37.473189 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.467928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:57127 (local address 127.17.198.62:38365)
0811 22:15:37.468097 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:37.468101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:37.469423 (+ 1322us) server_negotiation.cc:407] Connection header received
0811 22:15:37.470924 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:37.470927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:37.470977 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:37.471097 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:37.472583 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:37.472587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:37.472589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:15:37.472605 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:37.472631 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:15:37.472641 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:37.472747 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:37.472996 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 22:15:37.473049 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:15:37.474237 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.468404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38365 (local address 127.17.198.3:57127)
0811 22:15:37.469221 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:37.469272 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:37.470591 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:37.471379 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:37.471388 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:37.471494 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:37.472237 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:15:37.472239 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:37.472286 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:15:37.472366 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:37.472747 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:37.472754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:37.472759 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:15:37.473007 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 22:15:37.473248 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":125,"threads_started":1}
I20260811 22:15:37.475629 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29561
I20260811 22:15:37.475728 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:37.475997 29731 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38365
I20260811 22:15:37.476054 18200 external_mini_cluster.cc:1428] Setting key cb435bee86280d74bb7d35819b646a30
I20260811 22:15:37.476323 29731 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:37.476974 29731 heartbeater.cc:511] Master 127.17.198.62:38365 requested a full tablet report, sending...
I20260811 22:15:37.478173 29178 ts_manager.cc:194] Registered new tserver with Master: 84cbf66969744a12aec10da2f8ead9ad (127.17.198.3:40793)
I20260811 22:15:37.491264 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:15:37.494541 29227 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:15:37.493640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.17.198.62:38365)
0811 22:15:37.493809 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:37.493816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:37.493943 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:15:37.494052 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:37.494062 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:37.494102 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:37.494186 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:37.494416 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50656: BlockingRecv error: recv got EOF from 127.0.0.1:50656 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:15:37.494813 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29250
I20260811 22:15:37.509163 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29406
I20260811 22:15:37.518882 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29561
I20260811 22:15:37.529433 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29143
2026-08-11T22:15:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (990 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:37 dist-test-slave-77v9 krb5kdc[29742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:37 dist-test-slave-77v9 krb5kdc[29742](info): set up 2 sockets
Aug 11 22:15:37 dist-test-slave-77v9 krb5kdc[29742](info): commencing operation
krb5kdc: starting...
W20260811 22:15:38.702987 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:38 dist-test-slave-77v9 krb5kdc[29742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:15:38.857043 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:32835
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:32835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:38.978463 29756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:38.978827 29756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:38.978920 29756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:38.983309 29756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:38.983433 29756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:38.983466 29756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:38.983510 29756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:38.983542 29756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:38.989380 29756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:32835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:32835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:38.990578 29756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:38.991636 29756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.001186 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.001230 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.001255 29761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.001374 29756 server_base.cc:1034] running on GCE node
I20260811 22:15:39.002104 29756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.003327 29756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.004545 29756 hybrid_clock.cc:648] HybridClock initialized: now 1786486539004527 us; error 35 us; skew 500 ppm
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.007727 29756 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:15:39.008785 29756 webserver.cc:460] Webserver started at http://127.17.198.62:35355/ using document root <none> and password file <none>
I20260811 22:15:39.009461 29756 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.009565 29756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.009816 29756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.012253 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "bce556ae8d2b4375f5468e0676364d58"
server_key_iv: "4630db085d48e3f0716e3a38c2c2bfd6"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.012959 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "bce556ae8d2b4375f5468e0676364d58"
server_key_iv: "4630db085d48e3f0716e3a38c2c2bfd6"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.017289 29756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:15:39.020133 29773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.021859 29756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:39.022050 29756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "bce556ae8d2b4375f5468e0676364d58"
server_key_iv: "4630db085d48e3f0716e3a38c2c2bfd6"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.022195 29756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.040707 29756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.044561 29756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.044860 29756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.053555 29756 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:32835
I20260811 22:15:39.053882 29834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:32835 every 8 connection(s)
I20260811 22:15:39.054781 29756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:39.056172 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29756
I20260811 22:15:39.056300 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:15:39.056620 18200 external_mini_cluster.cc:1428] Setting key 96cf7c84a701695fdf6ca42c5c1c6772
I20260811 22:15:39.058588 29835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:39.066159 29835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: Bootstrap starting.
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:39.069247 29835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:39.070117 29835 log.cc:824] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:39.073043 29835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: No bootstrap required, opened a new log
I20260811 22:15:39.073776 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.058324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.17.198.62:32835)
0811 22:15:39.058800 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.058819 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.058871 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:15:39.059556 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.059574 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.059921 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.060217 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.061265 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.062360 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.063610 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.064146 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.064284 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.067754 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.067760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.067774 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.067813 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.070053 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.070705 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.070715 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.070722 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.070901 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.071333 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.071336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.071338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.071830 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.072015 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.072386 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.072747 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":167,"threads_started":1}
I20260811 22:15:39.077167 29835 raft_consensus.cc:348] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } }
I20260811 22:15:39.077507 29835 raft_consensus.cc:374] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:39.077579 29835 raft_consensus.cc:729] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c92ee8b3563f4d53a2e4b1747fef9d8f, State: Initialized, Role: FOLLOWER
I20260811 22:15:39.078051 29835 consensus_queue.cc:260] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } }
I20260811 22:15:39.078190 29835 raft_consensus.cc:388] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:39.078280 29835 raft_consensus.cc:482] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:39.078403 29835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:39.079501 29835 raft_consensus.cc:504] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } }
I20260811 22:15:39.079911 29835 leader_election.cc:302] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c92ee8b3563f4d53a2e4b1747fef9d8f; no voters:
I20260811 22:15:39.080286 29835 leader_election.cc:288] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:39.080500 29841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:39.080790 29841 raft_consensus.cc:686] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [term 1 LEADER]: Becoming Leader. State: Replica: c92ee8b3563f4d53a2e4b1747fef9d8f, State: Running, Role: LEADER
I20260811 22:15:39.081212 29841 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } }
I20260811 22:15:39.081965 29835 sys_catalog.cc:564] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:39.083138 29842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } } }
I20260811 22:15:39.083283 29842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:39.083606 29843 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92ee8b3563f4d53a2e4b1747fef9d8f. Latest consensus state: current_term: 1 leader_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92ee8b3563f4d53a2e4b1747fef9d8f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 32835 } } }
I20260811 22:15:39.083709 29843 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:39.083812 29850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:39.086871 29850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:39.092775 29850 catalog_manager.cc:1347] Generated new cluster ID: 09d6ad7f8d164522aee988718836d36b
I20260811 22:15:39.092903 29850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:39.106818 29850 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:39.107898 29850 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:39.123587 29850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: Generated new TSK 0
I20260811 22:15:39.124647 29850 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:15:39.164762 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:39.289065 29864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:39.289400 29864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:39.289450 29864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:39.293846 29864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:39.293931 29864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:39.294003 29864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:15:39.300050 29864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:39.301381 29864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:39.302490 29864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.313081 29872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.313220 29874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.313063 29871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.313397 29864 server_base.cc:1034] running on GCE node
I20260811 22:15:39.313918 29864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.314741 29864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.315929 29864 hybrid_clock.cc:648] HybridClock initialized: now 1786486539315893 us; error 53 us; skew 500 ppm
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.323423 29864 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:15:39.324568 29864 webserver.cc:460] Webserver started at http://127.17.198.1:41775/ using document root <none> and password file <none>
I20260811 22:15:39.325300 29864 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.325415 29864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.325726 29864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.328181 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "1196e6986d804eca985b0caa8750073d"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "febaec56e344849ab8b918f4f6d4b459"
server_key_iv: "004a5b30b50d18a1f5ff31686fdd3d13"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.328905 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "1196e6986d804eca985b0caa8750073d"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "febaec56e344849ab8b918f4f6d4b459"
server_key_iv: "004a5b30b50d18a1f5ff31686fdd3d13"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.333297 29864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:39.336325 29883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.337889 29864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:39.338054 29864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "1196e6986d804eca985b0caa8750073d"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "febaec56e344849ab8b918f4f6d4b459"
server_key_iv: "004a5b30b50d18a1f5ff31686fdd3d13"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.338193 29864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.355506 29864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.359444 29864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.359702 29864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.360388 29864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:39.361420 29864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:39.361503 29864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.361580 29864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:39.361624 29864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.371625 29864 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:36561
I20260811 22:15:39.371654 30013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:36561 every 8 connection(s)
I20260811 22:15:39.372601 29864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:39.374014 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 29864
I20260811 22:15:39.374150 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:39.374503 18200 external_mini_cluster.cc:1428] Setting key d490c67cc96eaeb0929332dedcfe9e73
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:15:39.389757 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.375002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:43301 (local address 127.17.198.62:32835)
0811 22:15:39.375154 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.375158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.376246 (+ 1088us) server_negotiation.cc:407] Connection header received
0811 22:15:39.377779 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.377783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.377843 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.377937 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.379169 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.379720 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.381090 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.381337 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.381458 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.384574 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.384577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.384580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.384610 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.386558 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.387697 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.387701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.387703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.387869 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.388340 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.388343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.388345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.388593 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.388724 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.389418 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.389580 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:15:39.390712 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.375506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:32835 (local address 127.17.198.1:43301)
0811 22:15:39.376067 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.376132 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.377557 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.378117 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.378125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.378520 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:39.379016 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.379030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.379863 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.379866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.380959 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.380969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.381512 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.381516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.381712 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.382388 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:39.382405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:39.384349 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:39.387192 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.387199 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.387215 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.387548 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.387982 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.387985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.387987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.388195 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.388725 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:39.388734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:39.389152 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.389374 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.389772 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":170,"threads_started":1}
I20260811 22:15:39.392398 30014 heartbeater.cc:347] Connected to a master server at 127.17.198.62:32835
I20260811 22:15:39.392709 30014 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:39.393317 30014 heartbeater.cc:511] Master 127.17.198.62:32835 requested a full tablet report, sending...
I20260811 22:15:39.395165 29791 ts_manager.cc:194] Registered new tserver with Master: 1196e6986d804eca985b0caa8750073d (127.17.198.1:36561)
I20260811 22:15:39.396453 29791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:43301
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:15:39.409454 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:39.533421 30019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:39.533759 30019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:39.533857 30019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:39.538213 30019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:39.538328 30019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:39.538429 30019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:15:39.544344 30019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:39.545606 30019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:39.546734 30019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.557186 30025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.557230 30019 server_base.cc:1034] running on GCE node
W20260811 22:15:39.557186 30024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.557188 30027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.558051 30019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.558763 30019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.559979 30019 hybrid_clock.cc:648] HybridClock initialized: now 1786486539559965 us; error 29 us; skew 500 ppm
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.563333 30019 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:15:39.564357 30019 webserver.cc:460] Webserver started at http://127.17.198.2:43539/ using document root <none> and password file <none>
I20260811 22:15:39.565044 30019 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.565128 30019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.565321 30019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.567669 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "1e1359d4af5442d084c84a117f5caab3"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "e620f7d3583760d5f9ad0dc4198c63e7"
server_key_iv: "f90387184d4ed0f1ab8216805c1b5d81"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.568339 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "1e1359d4af5442d084c84a117f5caab3"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "e620f7d3583760d5f9ad0dc4198c63e7"
server_key_iv: "f90387184d4ed0f1ab8216805c1b5d81"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.572991 30019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260811 22:15:39.575932 30035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.577838 30019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:15:39.578037 30019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "1e1359d4af5442d084c84a117f5caab3"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "e620f7d3583760d5f9ad0dc4198c63e7"
server_key_iv: "f90387184d4ed0f1ab8216805c1b5d81"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.578176 30019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.591622 30019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.595919 30019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.596196 30019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.597043 30019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:39.598208 30019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:39.598266 30019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.598347 30019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:39.598392 30019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.609058 30019 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:40861
I20260811 22:15:39.609141 30163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:40861 every 8 connection(s)
I20260811 22:15:39.610191 30019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:39.618748 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 30019
I20260811 22:15:39.618979 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:39.619297 18200 external_mini_cluster.cc:1428] Setting key cc0addf9721d4affd38727ee33a649cd
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:39.628693 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.612645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:43099 (local address 127.17.198.62:32835)
0811 22:15:39.612957 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.612962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.614005 (+ 1043us) server_negotiation.cc:407] Connection header received
0811 22:15:39.615687 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.615691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.615767 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.615861 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.617469 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.618087 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.619659 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.620068 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.620208 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.623671 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.623678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.623682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.623719 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.625608 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.626499 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.626503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.626505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.626595 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.627191 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.627197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.627202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.627434 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.627536 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.628418 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.628558 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:15:39.630246 30166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.613163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:32835 (local address 127.17.198.2:43099)
0811 22:15:39.613735 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.613790 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.615406 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.616077 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.616087 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.616535 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:39.617258 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.617283 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.618317 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.618320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.619510 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.619522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.620267 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.620272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.620483 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.621251 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:39.621269 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:39.623365 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:39.625782 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.625794 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.625827 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.626347 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.626715 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.626722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.626727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.626998 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.627566 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:39.627577 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:39.628033 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.628319 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.628849 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":151,"threads_started":1}
I20260811 22:15:39.631968 30164 heartbeater.cc:347] Connected to a master server at 127.17.198.62:32835
I20260811 22:15:39.632300 30164 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:15:39.633070 30164 heartbeater.cc:511] Master 127.17.198.62:32835 requested a full tablet report, sending...
I20260811 22:15:39.634462 29791 ts_manager.cc:194] Registered new tserver with Master: 1e1359d4af5442d084c84a117f5caab3 (127.17.198.2:40861)
I20260811 22:15:39.635154 29791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:43099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:15:39.657913 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:39.787431 30169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:39.787765 30169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:39.787847 30169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:39.792503 30169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:39.792595 30169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:39.792659 30169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:15:39.798702 30169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:32835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:15:39.799853 30169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:39.800971 30169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.811793 30177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.811846 30178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.811793 30180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.812099 30169 server_base.cc:1034] running on GCE node
I20260811 22:15:39.812724 30169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.813560 30169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.814785 30169 hybrid_clock.cc:648] HybridClock initialized: now 1786486539814741 us; error 61 us; skew 500 ppm
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.818193 30169 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:15:39.819198 30169 webserver.cc:460] Webserver started at http://127.17.198.3:36609/ using document root <none> and password file <none>
I20260811 22:15:39.819779 30169 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.819850 30169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.820044 30169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.822396 30169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "1a4b01b711014f549cf47f6540810b6a"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "c69053791cf2ceef45e92880ed06b436"
server_key_iv: "c478731c9b91578e866bf0c9701de493"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.822994 30169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "1a4b01b711014f549cf47f6540810b6a"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "c69053791cf2ceef45e92880ed06b436"
server_key_iv: "c478731c9b91578e866bf0c9701de493"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.827347 30169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:15:39.830139 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.831578 30169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:39.831698 30169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "1a4b01b711014f549cf47f6540810b6a"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-77v9"
server_key: "c69053791cf2ceef45e92880ed06b436"
server_key_iv: "c478731c9b91578e866bf0c9701de493"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.831867 30169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.865530 30169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.869356 30169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.869591 30169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.870321 30169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:39.871258 30169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:39.871309 30169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.871379 30169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:39.871414 30169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.883821 30169 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:46709
I20260811 22:15:39.884066 30320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:46709 every 8 connection(s)
I20260811 22:15:39.885149 30169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:39.887746 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 30169
I20260811 22:15:39.887893 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:39.888284 18200 external_mini_cluster.cc:1428] Setting key ecba795336d8e4c56fc302aac72c9e1c
Aug 11 22:15:39 dist-test-slave-77v9 krb5kdc[29742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:15:39.902525 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.887465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:49027 (local address 127.17.198.62:32835)
0811 22:15:39.887620 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.887624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.888922 (+ 1298us) server_negotiation.cc:407] Connection header received
0811 22:15:39.890668 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.890672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.890735 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.890804 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.892387 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.893127 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.894489 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.894713 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.894805 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.898058 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.898061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.898063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.898093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.899638 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.900236 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.900239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.900241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.900329 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.900737 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.900741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.900742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.901111 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.901228 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.902164 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.902318 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:15:39.903685 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.888007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:32835 (local address 127.17.198.3:49027)
0811 22:15:39.888634 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.888711 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.890409 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.891149 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.891158 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.891616 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:39.892193 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.892210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.893269 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.893272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.894342 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.894353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.894872 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.894874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.895089 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.895872 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:39.895900 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:39.897815 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:39.899790 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.899796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.899811 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.900115 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.900426 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.900428 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.900429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.900606 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.901306 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:39.901315 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:39.901723 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.902060 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.902547 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":181,"threads_started":1}
I20260811 22:15:39.904986 30321 heartbeater.cc:347] Connected to a master server at 127.17.198.62:32835
I20260811 22:15:39.905267 30321 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:39.905838 30321 heartbeater.cc:511] Master 127.17.198.62:32835 requested a full tablet report, sending...
I20260811 22:15:39.907003 29791 ts_manager.cc:194] Registered new tserver with Master: 1a4b01b711014f549cf47f6540810b6a (127.17.198.3:46709)
I20260811 22:15:39.907593 29791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:49027
I20260811 22:15:39.916164 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:39.927454 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.918738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.17.198.62:32835)
0811 22:15:39.918963 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.918969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.919093 (+ 124us) server_negotiation.cc:407] Connection header received
0811 22:15:39.919159 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.919161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.919211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.919296 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.920047 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.920667 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.921594 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.921841 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.921947 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.923483 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.923486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.923488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.923517 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.925312 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.925824 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.925828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.925829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.925990 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.926342 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.926370 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.926372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.926666 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.926788 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.927104 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.927241 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 22:15:39.931357 29791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:39.934806 29791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:39.952184 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.946114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:46709 (local address 127.0.0.1:33010)
0811 22:15:39.946829 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.946843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.946918 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.947664 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.947667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.947686 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:39.947809 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.947815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.949326 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.949333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.950362 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.950372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951459 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.951597 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.951641 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.951767 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.951997 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":13,"thread_start_us":113,"threads_started":1}
I20260811 22:15:39.952242 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.945434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.17.198.2:40861)
0811 22:15:39.946631 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.946643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.946667 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:15:39.946752 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.946758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.946989 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.947151 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:39.947486 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.948722 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.950819 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951472 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951585 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.951737 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.951948 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.952072 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":210,"threads_started":1}
I20260811 22:15:39.952291 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.946603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.17.198.3:46709)
0811 22:15:39.947196 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.947211 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.947232 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:15:39.947307 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.947314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.947495 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.947956 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:39.948119 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.949171 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.950501 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951305 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951427 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.951601 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.951997 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.952118 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":210,"threads_started":1}
I20260811 22:15:39.952606 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.945295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:40861 (local address 127.0.0.1:45520)
0811 22:15:39.946074 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.946131 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.946273 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.947150 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.947157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.947185 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:39.947347 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.947354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.948936 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.948940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.950656 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.950668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951691 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.951782 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.951804 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.951890 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.952451 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":563,"thread_start_us":86,"threads_started":1}
I20260811 22:15:39.952992 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.946168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.17.198.1:36561)
0811 22:15:39.947137 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.947144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.947166 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:15:39.947240 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.947244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.947446 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.947599 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:39.947863 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.948670 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.950808 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951595 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951701 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.951833 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.952255 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.952836 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":133,"threads_started":1}
I20260811 22:15:39.953162 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.945411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36561 (local address 127.0.0.1:52066)
0811 22:15:39.946398 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.946414 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.946533 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.947581 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.947584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.947602 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:39.947720 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.947730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.948799 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.948806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.950656 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.950668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.951899 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.951902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.952005 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.952026 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.952095 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.952161 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":129,"thread_start_us":132,"threads_started":1}
I20260811 22:15:39.956218 30245 tablet_service.cc:1467] Processing CreateTablet for tablet f79664a94ab948d887836b843aacfdba (DEFAULT_TABLE table=test-table [id=bed0a1ad2b9b449b98e6d50de5d126dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:39.956223 29936 tablet_service.cc:1467] Processing CreateTablet for tablet f79664a94ab948d887836b843aacfdba (DEFAULT_TABLE table=test-table [id=bed0a1ad2b9b449b98e6d50de5d126dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:39.956223 30088 tablet_service.cc:1467] Processing CreateTablet for tablet f79664a94ab948d887836b843aacfdba (DEFAULT_TABLE table=test-table [id=bed0a1ad2b9b449b98e6d50de5d126dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:39.957409 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79664a94ab948d887836b843aacfdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:39.957409 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79664a94ab948d887836b843aacfdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:39.957423 30245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79664a94ab948d887836b843aacfdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:39.965283 30342 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Bootstrap starting.
I20260811 22:15:39.966231 30345 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Bootstrap starting.
I20260811 22:15:39.966678 30344 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: Bootstrap starting.
I20260811 22:15:39.968353 30342 tablet_bootstrap.cc:654] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:39.968851 30344 tablet_bootstrap.cc:654] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:39.969103 30345 tablet_bootstrap.cc:654] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:39.969777 30342 log.cc:824] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:39.969806 30344 log.cc:824] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:39.969834 30345 log.cc:824] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:39.971913 30342 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: No bootstrap required, opened a new log
I20260811 22:15:39.971939 30344 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: No bootstrap required, opened a new log
I20260811 22:15:39.971939 30345 tablet_bootstrap.cc:492] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: No bootstrap required, opened a new log
I20260811 22:15:39.972152 30345 ts_tablet_manager.cc:1397] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:15:39.972152 30344 ts_tablet_manager.cc:1397] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:15:39.972157 30342 ts_tablet_manager.cc:1397] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 22:15:39.975770 30342 raft_consensus.cc:348] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.976102 30342 raft_consensus.cc:374] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:39.976192 30342 raft_consensus.cc:729] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4b01b711014f549cf47f6540810b6a, State: Initialized, Role: FOLLOWER
I20260811 22:15:39.976359 30345 raft_consensus.cc:348] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.976643 30345 raft_consensus.cc:374] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:39.976764 30345 raft_consensus.cc:729] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e1359d4af5442d084c84a117f5caab3, State: Initialized, Role: FOLLOWER
I20260811 22:15:39.976871 30342 consensus_queue.cc:260] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.976867 30344 raft_consensus.cc:348] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.977099 30344 raft_consensus.cc:374] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:39.977164 30344 raft_consensus.cc:729] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1196e6986d804eca985b0caa8750073d, State: Initialized, Role: FOLLOWER
I20260811 22:15:39.977252 30345 consensus_queue.cc:260] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.977567 30344 consensus_queue.cc:260] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:39.978157 30164 heartbeater.cc:503] Master 127.17.198.62:32835 was elected leader, sending a full tablet report...
I20260811 22:15:39.978360 30342 ts_tablet_manager.cc:1428] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:15:39.978361 30345 ts_tablet_manager.cc:1428] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 22:15:39.978528 30344 ts_tablet_manager.cc:1428] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:15:39.978698 30321 heartbeater.cc:503] Master 127.17.198.62:32835 was elected leader, sending a full tablet report...
I20260811 22:15:39.978924 30014 heartbeater.cc:503] Master 127.17.198.62:32835 was elected leader, sending a full tablet report...
I20260811 22:15:40.051549 30350 raft_consensus.cc:482] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:40.051806 30350 raft_consensus.cc:504] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:40.053344 30350 leader_election.cc:288] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e1359d4af5442d084c84a117f5caab3 (127.17.198.2:40861), 1196e6986d804eca985b0caa8750073d (127.17.198.1:36561)
I20260811 22:15:40.058753 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.054085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:51513 (local address 127.17.198.1:36561)
0811 22:15:40.054213 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:40.054219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:40.054233 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:15:40.054288 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:40.054293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:40.054346 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:40.054417 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:40.054866 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.055661 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.057595 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.058080 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.058235 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.058296 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:40.058524 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 22:15:40.058613 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:15:40.058759 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.053474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:40861 (local address 127.17.198.3:53401)
0811 22:15:40.053733 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:40.053754 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:40.053867 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:40.054398 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:40.054401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:40.054418 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:40.054570 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.054577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.055637 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.055643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.057389 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.057404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.058374 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.058377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.058507 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.058525 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:15:40.058594 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:15:40.058656 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":125}
I20260811 22:15:40.058768 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.053562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36561 (local address 127.17.198.3:51513)
0811 22:15:40.053984 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:40.053999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:40.054118 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:40.054502 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:40.054508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:40.054532 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:40.054718 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.054726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.055783 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.055786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.057405 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.057413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.058232 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.058236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.058358 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.058395 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:15:40.058485 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:15:40.058581 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":111,"thread_start_us":152,"threads_started":1}
I20260811 22:15:40.059355 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.053807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:53401 (local address 127.17.198.2:40861)
0811 22:15:40.054094 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:40.054099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:40.054116 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:40.054179 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:40.054182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:40.054241 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:40.054335 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:40.054743 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.055382 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.057562 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.058180 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.058320 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.058369 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:40.059094 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 22:15:40.059185 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260811 22:15:40.059854 29957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79664a94ab948d887836b843aacfdba" candidate_uuid: "1a4b01b711014f549cf47f6540810b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1196e6986d804eca985b0caa8750073d" is_pre_election: true
I20260811 22:15:40.059963 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79664a94ab948d887836b843aacfdba" candidate_uuid: "1a4b01b711014f549cf47f6540810b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e1359d4af5442d084c84a117f5caab3" is_pre_election: true
I20260811 22:15:40.060207 29957 raft_consensus.cc:2454] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4b01b711014f549cf47f6540810b6a in term 0.
I20260811 22:15:40.060250 30109 raft_consensus.cc:2454] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4b01b711014f549cf47f6540810b6a in term 0.
I20260811 22:15:40.060714 30193 leader_election.cc:302] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1196e6986d804eca985b0caa8750073d, 1a4b01b711014f549cf47f6540810b6a; no voters:
I20260811 22:15:40.061156 30350 raft_consensus.cc:2781] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:40.061254 30350 raft_consensus.cc:482] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:40.061347 30350 raft_consensus.cc:3037] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:40.062546 30350 raft_consensus.cc:504] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:40.063009 30350 leader_election.cc:288] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [CANDIDATE]: Term 1 election: Requested vote from peers 1e1359d4af5442d084c84a117f5caab3 (127.17.198.2:40861), 1196e6986d804eca985b0caa8750073d (127.17.198.1:36561)
I20260811 22:15:40.063436 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79664a94ab948d887836b843aacfdba" candidate_uuid: "1a4b01b711014f549cf47f6540810b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e1359d4af5442d084c84a117f5caab3"
I20260811 22:15:40.063469 29957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79664a94ab948d887836b843aacfdba" candidate_uuid: "1a4b01b711014f549cf47f6540810b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1196e6986d804eca985b0caa8750073d"
I20260811 22:15:40.063609 29957 raft_consensus.cc:3037] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:40.063614 30109 raft_consensus.cc:3037] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:40.064869 30109 raft_consensus.cc:2454] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4b01b711014f549cf47f6540810b6a in term 1.
I20260811 22:15:40.065032 29957 raft_consensus.cc:2454] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4b01b711014f549cf47f6540810b6a in term 1.
I20260811 22:15:40.065284 30191 leader_election.cc:302] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4b01b711014f549cf47f6540810b6a, 1e1359d4af5442d084c84a117f5caab3; no voters:
I20260811 22:15:40.065568 30350 raft_consensus.cc:2781] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:40.065899 30350 raft_consensus.cc:686] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 1 LEADER]: Becoming Leader. State: Replica: 1a4b01b711014f549cf47f6540810b6a, State: Running, Role: LEADER
I20260811 22:15:40.066327 30350 consensus_queue.cc:237] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } }
I20260811 22:15:40.070255 29790 catalog_manager.cc:5495] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a4b01b711014f549cf47f6540810b6a (127.17.198.3). New cstate: current_term: 1 leader_uuid: "1a4b01b711014f549cf47f6540810b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a4b01b711014f549cf47f6540810b6a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 46709 } health_report { overall_health: HEALTHY } } }
I20260811 22:15:40.083619 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.079461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.17.198.3:46709)
0811 22:15:40.079613 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:40.079620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:40.079639 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:40.079735 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:40.079738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:40.079801 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:40.079923 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:40.080188 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.080704 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.081844 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.082079 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.082266 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.082607 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:40.083105 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:40.083252 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:40.083362 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:15:40.083431 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:15:40.090253 30109 raft_consensus.cc:1261] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 1 FOLLOWER]: Refusing update from remote peer 1a4b01b711014f549cf47f6540810b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:40.090253 29957 raft_consensus.cc:1261] T f79664a94ab948d887836b843aacfdba P 1196e6986d804eca985b0caa8750073d [term 1 FOLLOWER]: Refusing update from remote peer 1a4b01b711014f549cf47f6540810b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:40.091012 30354 consensus_queue.cc:1035] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e1359d4af5442d084c84a117f5caab3" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 40861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:40.091331 30350 consensus_queue.cc:1035] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1196e6986d804eca985b0caa8750073d" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:40.103154 30359 mvcc.cc:205] Tried to move back new op lower bound from 7317448868201009152 to 7317448868115324928. Current Snapshot: MvccSnapshot[applied={T|T < 7317448868201009152}]
I20260811 22:15:40.109848 29790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:40.110090 29790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:40.113166 29790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c92ee8b3563f4d53a2e4b1747fef9d8f: Sending DeleteTablet for 3 replicas of tablet f79664a94ab948d887836b843aacfdba
I20260811 22:15:40.114185 30088 tablet_service.cc:1514] Processing DeleteTablet for tablet f79664a94ab948d887836b843aacfdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:40 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:45520
I20260811 22:15:40.114243 29936 tablet_service.cc:1514] Processing DeleteTablet for tablet f79664a94ab948d887836b843aacfdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:40 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:52066
I20260811 22:15:40.114810 30245 tablet_service.cc:1514] Processing DeleteTablet for tablet f79664a94ab948d887836b843aacfdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:40 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:33010
I20260811 22:15:40.114854 30367 tablet_replica.cc:330] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: stopping tablet replica
I20260811 22:15:40.115264 30367 raft_consensus.cc:2229] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:40.115311 30371 tablet_replica.cc:330] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: stopping tablet replica
I20260811 22:15:40.115671 30367 raft_consensus.cc:2258] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:40.117671 30367 ts_tablet_manager.cc:1905] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:40.118139 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29864
I20260811 22:15:40.120414 30371 raft_consensus.cc:2229] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:40.121393 30371 raft_consensus.cc:2258] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:40.121572 30367 ts_tablet_manager.cc:1918] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:40.121706 30367 log.cc:1196] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/f79664a94ab948d887836b843aacfdba
I20260811 22:15:40.122107 30367 ts_tablet_manager.cc:1939] T f79664a94ab948d887836b843aacfdba P 1e1359d4af5442d084c84a117f5caab3: Deleting consensus metadata
I20260811 22:15:40.123720 29775 catalog_manager.cc:4841] TS 1e1359d4af5442d084c84a117f5caab3 (127.17.198.2:40861): tablet f79664a94ab948d887836b843aacfdba (table test-table [id=bed0a1ad2b9b449b98e6d50de5d126dc]) successfully deleted
I20260811 22:15:40.125324 30371 ts_tablet_manager.cc:1905] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:40.129599 30371 ts_tablet_manager.cc:1918] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:40.129896 30371 log.cc:1196] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/f79664a94ab948d887836b843aacfdba
I20260811 22:15:40.130386 30371 ts_tablet_manager.cc:1939] T f79664a94ab948d887836b843aacfdba P 1a4b01b711014f549cf47f6540810b6a: Deleting consensus metadata
W20260811 22:15:40.132028 29777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:36561 (error 108)
I20260811 22:15:40.132095 29774 catalog_manager.cc:4841] TS 1a4b01b711014f549cf47f6540810b6a (127.17.198.3:46709): tablet f79664a94ab948d887836b843aacfdba (table test-table [id=bed0a1ad2b9b449b98e6d50de5d126dc]) successfully deleted
I20260811 22:15:40.132309 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 30019
I20260811 22:15:40.142387 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.141917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36561 (local address 127.0.0.1:52072)
0811 22:15:40.142166 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:40.142266 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 22:15:40.142632 29777 catalog_manager.cc:4584] TS 1196e6986d804eca985b0caa8750073d (127.17.198.1:36561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f79664a94ab948d887836b843aacfdba: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36561: connect: Connection refused (error 111)
I20260811 22:15:40.148128 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 30169
I20260811 22:15:40.157879 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 29756
2026-08-11T22:15:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2628 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:40 dist-test-slave-77v9 krb5kdc[30377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:40 dist-test-slave-77v9 krb5kdc[30377](info): set up 2 sockets
Aug 11 22:15:40 dist-test-slave-77v9 krb5kdc[30377](info): commencing operation
krb5kdc: starting...
W20260811 22:15:41.361272 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:15:41 dist-test-slave-77v9 krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/rangeradmin_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/rangeradmin_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/rangerlookup_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/rangerlookup_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
I20260811 22:15:41.573124 18200 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taski7rXue/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/postgres -l logfile start
2026-08-11 22:15:44.948 UTC [30412] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:15:44.948 UTC [30412] LOG: listening on IPv4 address "127.17.198.20", port 57635
2026-08-11 22:15:44.953 UTC [30412] LOG: listening on Unix socket "/tmp/.s.PGSQL.57635"
2026-08-11 22:15:44.978 UTC [30415] LOG: database system was shut down at 2026-08-11 22:15:42 UTC
2026-08-11 22:15:44.983 UTC [30412] LOG: database system is ready to accept connections
I20260811 22:15:46.026594 18200 mini_postgres.cc:96] Postgres bound to 57635
2026-08-11 22:15:46.039 UTC [30424] FATAL: database "slave" does not exist
127.17.198.20:57635 - accepting connections
I20260811 22:15:46.040967 18200 mini_ranger.cc:161] Starting Ranger...
I20260811 22:15:46.061573 18200 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:15:46.716418 18200 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:15:46.716526 18200 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin
2026-08-11 22:15:47,209 [I] DB FLAVOR :POSTGRES
2026-08-11 22:15:47,210 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:15:47,210 [I] Checking connection..
2026-08-11 22:15:47,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:15:47,531 [I] Checking connection passed.
2026-08-11 22:15:47,531 [I] --------- Verifying version history table ---------
2026-08-11 22:15:47,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:15:47,814 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:15:47,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:15:48,094 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:15:48,094 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:15:48,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:15:48.377 UTC [30499] WARNING: there is no transaction in progress
2026-08-11 22:15:48,387 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:15:48,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:15:48,665 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:15:48,665 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:15:48,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:15:48,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:15:49,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-77v9.c.gcp-upstream.internal','N') ;"
2026-08-11 22:15:49,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:15:49,782 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:15:49,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:15:50,080 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:15:50,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:15:50,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:15:50,634 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:15:50,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:15:51.838 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:51.860 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:51.880 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:51.900 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:51.923 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:51.949 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.026 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.035 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.054 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.067 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.079 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.098 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.112 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.121 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.130 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:52.139 UTC [30631] WARNING: there is no transaction in progress
2026-08-11 22:15:53,038 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:15:53,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:15:53,288 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:15:53,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:15:53,583 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:15:53,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:15:53,872 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:15:53,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:15:54,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:15:54,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-77v9.c.gcp-upstream.internal';"
2026-08-11 22:15:54,675 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:15:54,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:57635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:15:54,961 [I] DB_PATCHES have already been applied
I20260811 22:15:54.967773 18200 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taski7rXue/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:15:54.967854 18200 mini_ranger.cc:193] Using host: 127.17.198.20
I20260811 22:15:54.972680 18200 mini_ranger.cc:227] Ranger admin URL: http://127.17.198.20:57127
Aug 11, 2026 10:15:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:15:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:15:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:15:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:15:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
Aug 11, 2026 10:16:29 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:16:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIs
Aug 11, 2026 10:16:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:16:29 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:16:30 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:17:56.123445 18200 mini_ranger.cc:160] Time spent starting Ranger: real 130.082s user 0.036s sys 0.178s
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:1013: Failure
Failed
Bad status: Timed out: Failed to start the Ranger service: process with PID 30740 is not yet bound to any port at the specified addresses; last attempt running lsof returned 'Not found: could not find pattern of a bound port in lsof output: p30740
f310
n127.0.0.1:44610->127.17.198.20:57635
f311
n127.0.0.1:44612->127.17.198.20:57635
f312
n127.0.0.1:44638->127.17.198.20:57635
f313
n127.0.0.1:44644->127.17.198.20:57635
f314
n127.0.0.1:44650->127.17.198.20:57635'
I20260811 22:17:56.124452 18200 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:17:56.166635 18200 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:17:56.167 UTC [30412] LOG: received smart shutdown request
2026-08-11 22:17:56.172 UTC [30412] LOG: background worker "logical replication launcher" (PID 30422) exited with exit code 1
2026-08-11 22:17:56.172 UTC [30417] LOG: shutting down
2026-08-11 22:17:56.419 UTC [30412] LOG: database system is shut down
2026-08-11T22:17:56Z chronyd exiting
I20260811 22:17:56.433053 18200 test_util.cc:175] -----------------------------------------------
I20260811 22:17:56.433144 18200 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486470476166-18200-0
[ FAILED ] SecurityITest.TestEncryptionWithKMSIntegration (136250 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:56 dist-test-slave-77v9 krb5kdc[30872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:56 dist-test-slave-77v9 krb5kdc[30872](info): set up 2 sockets
Aug 11 22:17:56 dist-test-slave-77v9 krb5kdc[30872](info): commencing operation
krb5kdc: starting...
W20260811 22:17:57.559618 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:17:57 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangeradmin_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangeradmin_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangerlookup_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangerlookup_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
I20260811 22:17:57.773334 18200 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taski7rXue/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/postgres -l logfile start
2026-08-11 22:18:00.952 UTC [30910] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:18:00.952 UTC [30910] LOG: listening on IPv4 address "127.17.198.20", port 60005
2026-08-11 22:18:00.958 UTC [30910] LOG: listening on Unix socket "/tmp/.s.PGSQL.60005"
2026-08-11 22:18:00.984 UTC [30913] LOG: database system was shut down at 2026-08-11 22:17:58 UTC
2026-08-11 22:18:00.989 UTC [30910] LOG: database system is ready to accept connections
I20260811 22:18:02.025723 18200 mini_postgres.cc:96] Postgres bound to 60005
2026-08-11 22:18:02.034 UTC [30921] FATAL: database "slave" does not exist
127.17.198.20:60005 - accepting connections
I20260811 22:18:02.034965 18200 mini_ranger.cc:161] Starting Ranger...
I20260811 22:18:02.056108 18200 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:18:02.717069 18200 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:18:02.717175 18200 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin
2026-08-11 22:18:03,052 [I] DB FLAVOR :POSTGRES
2026-08-11 22:18:03,053 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:18:03,053 [I] Checking connection..
2026-08-11 22:18:03,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:18:03,333 [I] Checking connection passed.
2026-08-11 22:18:03,333 [I] --------- Verifying version history table ---------
2026-08-11 22:18:03,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:18:03,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:18:03,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:18:03,919 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:18:03,919 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:18:03,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:18:04.244 UTC [30994] WARNING: there is no transaction in progress
2026-08-11 22:18:04,252 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:18:04,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:18:04,524 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:18:04,525 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:18:04,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:18:04,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:18:05,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-77v9.c.gcp-upstream.internal','N') ;"
2026-08-11 22:18:05,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:18:05,630 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:18:05,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:18:05,900 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:18:05,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:18:06,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:18:06,478 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:18:06,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:18:07.664 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.687 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.710 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.729 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.749 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.771 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.840 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.850 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.867 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.880 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.893 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.911 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.922 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.931 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.942 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:07.952 UTC [31120] WARNING: there is no transaction in progress
2026-08-11 22:18:08,837 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:18:08,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:18:09,109 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:18:09,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:18:09,379 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:18:09,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:18:09,672 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:18:09,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:18:09,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:18:10,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-77v9.c.gcp-upstream.internal';"
2026-08-11 22:18:10,481 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:18:10,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:18:10,752 [I] DB_PATCHES have already been applied
I20260811 22:18:10.758817 18200 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taski7rXue/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:18:10.758895 18200 mini_ranger.cc:193] Using host: 127.17.198.20
I20260811 22:18:10.763355 18200 mini_ranger.cc:227] Ranger admin URL: http://127.17.198.20:45341
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:18:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:18:12.106793 18200 mini_ranger.cc:160] Time spent starting Ranger: real 10.072s user 0.000s sys 0.015s
Aug 11, 2026 10:18:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:18:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 10:18:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:18:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:18:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:18:35.801352 18200 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.198.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.198.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangerkms_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.198.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/rangerkms_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.198.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.198.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.198.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/HTTP_127.17.198.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:18:35 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:18:35.905581 18200 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:18:35.917619 18200 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:18:36.983325 18200 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:18:36.983455 18200 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms
I20260811 22:18:36.983592 18200 mini_ranger_kms.cc:227] Using postgres at 127.17.198.20:60005
2026-08-11 22:18:37,543 [I] DB FLAVOR :POSTGRES
2026-08-11 22:18:37,544 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:18:37,544 [I] Checking connection
2026-08-11 22:18:37,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:18:37,839 [I] connection success
2026-08-11 22:18:37,840 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:18:37,840 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:18:37,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:18:38,135 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:18:38,135 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:18:38,136 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:18:38,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.198.20:60005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:18:38,444 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:18:39.067945 18200 mini_ranger_kms.cc:310] Created kms service
I20260811 22:18:39.316042 18200 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:18:39.373072 18200 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:18:39 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:39.876260 18200 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:18:39.876436 18200 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms:/tmp/dist-test-taski7rXue/build/debug/bin/postgresql.jar:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taski7rXue/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:18:39.876461 18200 mini_ranger_kms.cc:240] Using host: 127.17.198.20
I20260811 22:18:39.882273 18200 mini_ranger_kms.cc:276] Ranger KMS PID: 31340
I20260811 22:18:39.882472 18200 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.198.20:48629
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 22:18:42.431351 18200 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.526s user 0.004s sys 0.449s
I20260811 22:18:42.431504 18200 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:18:42.431617 18200 mini_ranger_kms.cc:401] 127.17.198.20:48629/kms/v1/keys
Aug 11 22:18:42 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.198.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:18:43 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.198.20@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
Aug 11 22:18:43 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.198.20@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
Aug 11 22:18:46 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:18:46.329202 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45407
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:45407
--ranger_config_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:46.482583 31400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:46.482913 31400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:46.482955 31400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:46.487447 31400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:46.487545 31400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:46.487565 31400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:46.487583 31400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:46.487599 31400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:46.495097 31400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.198.20:48629/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:45407
--ranger_config_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:46.496471 31400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:46.497745 31400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:46.509492 31406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:46.509585 31409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:46.509430 31407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:46.510260 31400 server_base.cc:1034] running on GCE node
I20260811 22:18:46.511036 31400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:46.512677 31400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:46.513912 31400 hybrid_clock.cc:648] HybridClock initialized: now 1786486726513879 us; error 58 us; skew 500 ppm
Aug 11 22:18:46 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:46.524217 31400 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:18:46.525480 31400 webserver.cc:460] Webserver started at http://127.17.198.62:36131/ using document root <none> and password file <none>
I20260811 22:18:46.526156 31400 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:46.526253 31400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:46.526530 31400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:18:46 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:46.604060 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "b8e7d9c69c8f4810832df62cfcd22626"
format_stamp: "Formatted at 2026-08-11 22:18:46 on dist-test-slave-77v9"
server_key: "9235c7a7e262a370e8836c969eef8d33"
server_key_iv: "f1aab9abe944fc55f68011c9b7bda1bf"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:46.604959 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "b8e7d9c69c8f4810832df62cfcd22626"
format_stamp: "Formatted at 2026-08-11 22:18:46 on dist-test-slave-77v9"
server_key: "9235c7a7e262a370e8836c969eef8d33"
server_key_iv: "f1aab9abe944fc55f68011c9b7bda1bf"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:46.609897 31400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:46.630228 31417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:46.631804 31400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:46.631981 31400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "b8e7d9c69c8f4810832df62cfcd22626"
format_stamp: "Formatted at 2026-08-11 22:18:46 on dist-test-slave-77v9"
server_key: "9235c7a7e262a370e8836c969eef8d33"
server_key_iv: "f1aab9abe944fc55f68011c9b7bda1bf"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:46.632128 31400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:46.651950 31400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:46.655498 31400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:46.655730 31400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:46.668454 31400 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:45407
I20260811 22:18:46.668509 31480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:45407 every 8 connection(s)
I20260811 22:18:46.669804 31400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:46.672165 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 31400
I20260811 22:18:46.672300 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:18:46.672637 18200 external_mini_cluster.cc:1428] Setting key b81fed8dc848895ac2a946bcb4c5a719
I20260811 22:18:46.673450 31481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:46.681617 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Bootstrap starting.
Aug 11 22:18:46 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:46.684741 31481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:46.685806 31481 log.cc:824] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:46.688318 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: No bootstrap required, opened a new log
I20260811 22:18:46.692471 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:46.674335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.17.198.62:45407)
0811 22:18:46.674974 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:46.674996 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:46.675040 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:18:46.676052 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:46.676079 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:46.676549 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:46.676975 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:46.678061 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:46.679668 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:46.680960 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:46.681698 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:46.681955 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:46.685431 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:46.685439 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:46.685459 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:46.685498 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:46.688600 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:46.689334 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:46.689341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:46.689348 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:46.689501 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:46.689861 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:46.689864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:46.689866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:46.690465 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:46.690760 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:46.690994 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:18:46.691329 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":225,"threads_started":1}
I20260811 22:18:46.692593 31481 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:46.692953 31481 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:46.693055 31481 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e7d9c69c8f4810832df62cfcd22626, State: Initialized, Role: FOLLOWER
I20260811 22:18:46.693755 31481 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:46.694058 31481 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:46.694123 31481 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:46.694211 31481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:46.695533 31481 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:46.695938 31487 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:46.695943 31481 leader_election.cc:302] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8e7d9c69c8f4810832df62cfcd22626; no voters:
I20260811 22:18:46.696130 31487 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:46.696508 31481 leader_election.cc:288] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:46.696666 31488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:46.696842 31487 leader_election.cc:302] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8e7d9c69c8f4810832df62cfcd22626; no voters:
I20260811 22:18:46.697022 31488 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 LEADER]: Becoming Leader. State: Replica: b8e7d9c69c8f4810832df62cfcd22626, State: Running, Role: LEADER
I20260811 22:18:46.697171 31487 leader_election.cc:288] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260811 22:18:46.697422 31488 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:46.698088 31481 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:46.699275 31487 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8e7d9c69c8f4810832df62cfcd22626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } } }
I20260811 22:18:46.699412 31487 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:46.700618 31491 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8e7d9c69c8f4810832df62cfcd22626. Latest consensus state: current_term: 1 leader_uuid: "b8e7d9c69c8f4810832df62cfcd22626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } } }
I20260811 22:18:46.700730 31491 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:46.701243 31497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:46.704777 31497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:46.705396 31481 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:18:46.723656 31497 catalog_manager.cc:1347] Generated new cluster ID: 548c86d90b2045a292970cb2e3b78821
I20260811 22:18:46.723923 31497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:46.737567 31497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:46.738992 31497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:46.752977 31497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Generated new TSK 0
I20260811 22:18:46.753968 31497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:18:47.753407 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:47.967305 31534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:47.967643 31534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:47.967695 31534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:47.974735 31534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:47.974870 31534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:47.975000 31534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:18:47.984571 31534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:47.986476 31534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:47.987982 31534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:48.002588 31543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.002836 31542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.002717 31545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.004410 31534 server_base.cc:1034] running on GCE node
I20260811 22:18:48.005080 31534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:48.006031 31534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:48.009395 31534 hybrid_clock.cc:648] HybridClock initialized: now 1786486728009315 us; error 84 us; skew 500 ppm
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:48.014768 31534 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:18:48.016546 31534 webserver.cc:460] Webserver started at http://127.17.198.1:41365/ using document root <none> and password file <none>
I20260811 22:18:48.017529 31534 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:48.017746 31534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:48.018100 31534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:48.042239 31534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "9019b24563bb4cbab82b1294b1db7e76"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "11c526f84d9bfe04467499dd53e55595"
server_key_iv: "ef9642ae786ef88739bacb6a753d55ea"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.043212 31534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "9019b24563bb4cbab82b1294b1db7e76"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "11c526f84d9bfe04467499dd53e55595"
server_key_iv: "ef9642ae786ef88739bacb6a753d55ea"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.050679 31534 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260811 22:18:48.079407 31553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.081449 31534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:48.081645 31534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "9019b24563bb4cbab82b1294b1db7e76"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "11c526f84d9bfe04467499dd53e55595"
server_key_iv: "ef9642ae786ef88739bacb6a753d55ea"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.081827 31534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:48.111413 31534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:48.115967 31534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:48.116225 31534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:48.117113 31534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:48.118496 31534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:48.118595 31534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.118693 31534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:48.118757 31534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.139750 31534 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:38355
I20260811 22:18:48.141487 31534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:48.142729 31692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:38355 every 8 connection(s)
I20260811 22:18:48.146347 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 31534
I20260811 22:18:48.146539 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:48.146935 18200 external_mini_cluster.cc:1428] Setting key 3bef0cd267b1d42e6c5eb3f779cf7fbf
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:18:48.171056 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.148573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:47545 (local address 127.17.198.62:45407)
0811 22:18:48.153105 (+ 4532us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:48.153110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:48.155181 (+ 2071us) server_negotiation.cc:407] Connection header received
0811 22:18:48.155238 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:48.155241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:48.155305 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:48.155393 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:48.157423 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.158316 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.160181 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.160565 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.160649 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.165377 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:48.165381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:48.165384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:48.165418 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:48.167929 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.168741 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.168745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.168747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.168895 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.169385 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.169388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.169390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.169673 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:48.169767 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:48.170679 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 22:18:48.170844 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4423,"thread_start_us":108,"threads_started":1}
I20260811 22:18:48.172384 31695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.146944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.1:47545)
0811 22:18:48.153477 (+ 6533us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:48.153570 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:48.154826 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:48.155746 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:48.155757 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:48.156354 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:48.157203 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.157224 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.158508 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.158511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.160012 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.160026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.160929 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.160934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.161173 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.162441 (+ 1268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:48.162475 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:48.165093 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:48.168114 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.168122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.168144 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.168594 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.169025 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.169028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.169030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.169255 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.169859 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:48.169867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:48.170263 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 22:18:48.170541 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 22:18:48.171111 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6243,"thread_start_us":244,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:18:48.174316 31693 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:48.174633 31693 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:48.175381 31693 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:48.177729 31437 ts_manager.cc:194] Registered new tserver with Master: 9019b24563bb4cbab82b1294b1db7e76 (127.17.198.1:38355)
I20260811 22:18:48.179747 31437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:47545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:18:48.201305 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:18:48.401743 31700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:48.402108 31700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:48.402185 31700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:48.407047 31700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:48.407167 31700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:48.407279 31700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:18:48.413486 31700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:48.414746 31700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:48.416086 31700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:48.428223 31706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.428784 31700 server_base.cc:1034] running on GCE node
W20260811 22:18:48.428714 31707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.429209 31709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.429716 31700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:48.430447 31700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:48.431638 31700 hybrid_clock.cc:648] HybridClock initialized: now 1786486728431606 us; error 44 us; skew 500 ppm
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:48.435065 31700 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:18:48.436192 31700 webserver.cc:460] Webserver started at http://127.17.198.2:42897/ using document root <none> and password file <none>
I20260811 22:18:48.437081 31700 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:48.437194 31700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:48.437433 31700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:48.456995 31700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "0c3fa8a893ff43aa94472fc257cbd6ff"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "4c559b5f984c8d06ff8de65b532d0066"
server_key_iv: "e5b0fa734fe0ac8186d93fa2534c2ffb"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.457963 31700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "0c3fa8a893ff43aa94472fc257cbd6ff"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "4c559b5f984c8d06ff8de65b532d0066"
server_key_iv: "e5b0fa734fe0ac8186d93fa2534c2ffb"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.464547 31700 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:18:48.485481 31717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.487599 31700 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:18:48.488003 31700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "0c3fa8a893ff43aa94472fc257cbd6ff"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "4c559b5f984c8d06ff8de65b532d0066"
server_key_iv: "e5b0fa734fe0ac8186d93fa2534c2ffb"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.488209 31700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:48.525309 31700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:48.530019 31700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:48.530309 31700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:48.531351 31700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:48.533047 31700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:48.533126 31700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.533193 31700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:48.533226 31700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.550904 31700 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43083
I20260811 22:18:48.550987 31846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43083 every 8 connection(s)
I20260811 22:18:48.552347 31700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:48.554960 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 31700
I20260811 22:18:48.555100 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:48.555457 18200 external_mini_cluster.cc:1428] Setting key 667fb175b266a72cd5a7cc7179072a4c
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:48.584892 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.554408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:46003 (local address 127.17.198.62:45407)
0811 22:18:48.554576 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:48.554581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:48.561547 (+ 6966us) server_negotiation.cc:407] Connection header received
0811 22:18:48.562840 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:48.562844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:48.562911 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:48.562992 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:48.565135 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.566121 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.568225 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.568593 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.568681 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.575936 (+ 7255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:48.575942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:48.575944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:48.575983 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:48.578232 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.581479 (+ 3247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.581485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.581487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.581614 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.582224 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.582228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.582230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.582550 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:48.582675 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:48.584476 (+ 1801us) server_negotiation.cc:299] Negotiation successful
0811 22:18:48.584654 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:18:48.587100 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.554932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.2:46003)
0811 22:18:48.561312 (+ 6380us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:48.561404 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:48.562526 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:48.563498 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:48.563509 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:48.563952 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:48.564917 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.564946 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.566354 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.566357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.568064 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.568076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.569138 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.569146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.569427 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.570306 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:48.570331 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:48.575594 (+ 5263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:48.580463 (+ 4869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.580484 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.580517 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.581264 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.581788 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.581791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.581794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.582084 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.582759 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:48.582780 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:48.584085 (+ 1305us) client_negotiation.cc:769] Sending connection context
0811 22:18:48.585447 (+ 1362us) client_negotiation.cc:240] Negotiation successful
0811 22:18:48.585890 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6051,"thread_start_us":163,"threads_started":1}
I20260811 22:18:48.589066 31847 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:48.589367 31847 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:48.589998 31847 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:48.591365 31437 ts_manager.cc:194] Registered new tserver with Master: 0c3fa8a893ff43aa94472fc257cbd6ff (127.17.198.2:43083)
I20260811 22:18:48.592144 31437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:46003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:18:48.601648 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:48.800338 31852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:48.800931 31852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:48.801077 31852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:48.808264 31852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:48.808394 31852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:48.808492 31852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:18:48.817550 31852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:48.819336 31852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:48.820995 31852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:48.844679 31860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.845332 31857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.846235 31858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.846604 31852 server_base.cc:1034] running on GCE node
I20260811 22:18:48.847270 31852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:48.848501 31852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:48.851799 31852 hybrid_clock.cc:648] HybridClock initialized: now 1786486728851707 us; error 94 us; skew 500 ppm
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:48.856802 31852 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:18:48.858587 31852 webserver.cc:460] Webserver started at http://127.17.198.3:33381/ using document root <none> and password file <none>
I20260811 22:18:48.859489 31852 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:48.859670 31852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:48.860005 31852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:18:48 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:48.895625 31852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "ada79ffc4a2d4b2ba194ef313b4e212c"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "8a81759b2e9a7fc5abe7254ee09eabdf"
server_key_iv: "9852f1c29ac0867af17d68f159254d31"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.896869 31852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "ada79ffc4a2d4b2ba194ef313b4e212c"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "8a81759b2e9a7fc5abe7254ee09eabdf"
server_key_iv: "9852f1c29ac0867af17d68f159254d31"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.905807 31852 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.003s
I20260811 22:18:48.934540 31868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.936305 31852 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:48.936452 31852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "ada79ffc4a2d4b2ba194ef313b4e212c"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "8a81759b2e9a7fc5abe7254ee09eabdf"
server_key_iv: "9852f1c29ac0867af17d68f159254d31"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:48.936591 31852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:48.982885 31852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:48.986696 31852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:48.986935 31852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:48.987638 31852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:48.988940 31852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:48.989012 31852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.989087 31852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:48.989120 31852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.009207 31852 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:41289
I20260811 22:18:49.009241 31996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:41289 every 8 connection(s)
I20260811 22:18:49.010326 31852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:49.019181 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 31852
I20260811 22:18:49.019343 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:49.019687 18200 external_mini_cluster.cc:1428] Setting key a0ab5fb104b055ef81cd0f64cab481f5
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:49.032356 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.013367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:56731 (local address 127.17.198.62:45407)
0811 22:18:49.013574 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.013588 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.014763 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 22:18:49.016830 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.016857 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.016930 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.017024 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.019753 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.020523 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.022925 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.023178 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.023289 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.027256 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.027261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.027264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.027300 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.029332 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.030165 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.030169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.030172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.030298 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.030812 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.030818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.030821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.031117 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.031237 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.031993 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.032170 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:18:49.033780 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.013667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.3:56731)
0811 22:18:49.014420 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.014539 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.016396 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.017381 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.017392 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.018175 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:49.019507 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.019548 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.020692 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.020696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.022743 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.022759 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.023376 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.023380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.023728 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.024641 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:49.024665 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:49.026981 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:49.029529 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.029537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.029560 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.030018 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.030434 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.030437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.030439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.030664 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.031254 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:49.031262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:49.031647 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.031891 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.032280 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":148,"threads_started":1}
I20260811 22:18:49.035604 31997 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:49.035924 31997 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.036648 31997 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:49.038069 31437 ts_manager.cc:194] Registered new tserver with Master: ada79ffc4a2d4b2ba194ef313b4e212c (127.17.198.3:41289)
I20260811 22:18:49.038892 31437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:56731
I20260811 22:18:49.047354 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:18:49.048190 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 31534
I20260811 22:18:49.060850 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 31700
I20260811 22:18:49.068857 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 31852
I20260811 22:18:49.077806 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 31400
I20260811 22:18:49.087692 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45407
--webserver_interface=127.17.198.62
--webserver_port=36131
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:45407
--ranger_config_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:49.204970 32000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:49.205312 32000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:49.205356 32000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:49.209793 32000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:49.209892 32000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:49.209911 32000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:49.209928 32000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:49.209944 32000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:49.215963 32000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:45407
--ranger_config_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=36131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:49.217178 32000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:49.218214 32000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:49.227994 32006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.228006 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:49.228070 32000 server_base.cc:1034] running on GCE node
W20260811 22:18:49.228010 32007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:49.229012 32000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:49.230252 32000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:49.231446 32000 hybrid_clock.cc:648] HybridClock initialized: now 1786486729231400 us; error 66 us; skew 500 ppm
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:49.234843 32000 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:18:49.236047 32000 webserver.cc:460] Webserver started at http://127.17.198.62:36131/ using document root <none> and password file <none>
I20260811 22:18:49.236685 32000 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:49.236797 32000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:49.319922 32000 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:18:49.322599 32020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.324018 32000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:18:49.324183 32000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "b8e7d9c69c8f4810832df62cfcd22626"
format_stamp: "Formatted at 2026-08-11 22:18:46 on dist-test-slave-77v9"
server_key: "9235c7a7e262a370e8836c969eef8d33"
server_key_iv: "f1aab9abe944fc55f68011c9b7bda1bf"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:49.324776 32000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:49.335228 32000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:49.338644 32000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:49.338876 32000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:49.346683 32000 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:45407
I20260811 22:18:49.346745 32079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:45407 every 8 connection(s)
I20260811 22:18:49.347991 32000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:49.353251 32080 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:18:49.355310 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Bootstrap starting.
I20260811 22:18:49.358178 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32000
I20260811 22:18:49.358721 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:38355
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:45407
--webserver_port=41365
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:18:49.373847 32080 log.cc:824] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:49.381394 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:18:49.381881 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Bootstrap complete.
I20260811 22:18:49.386272 32080 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:49.386756 32080 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e7d9c69c8f4810832df62cfcd22626, State: Initialized, Role: FOLLOWER
I20260811 22:18:49.387284 32080 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:49.387457 32080 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:49.387540 32080 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:49.387660 32080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:18:49.389971 32080 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:49.390342 32080 leader_election.cc:302] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8e7d9c69c8f4810832df62cfcd22626; no voters:
I20260811 22:18:49.390717 32080 leader_election.cc:288] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:18:49.390822 32085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:18:49.391096 32085 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [term 2 LEADER]: Becoming Leader. State: Replica: b8e7d9c69c8f4810832df62cfcd22626, State: Running, Role: LEADER
I20260811 22:18:49.391510 32085 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } }
I20260811 22:18:49.392091 32080 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:49.392973 32086 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b8e7d9c69c8f4810832df62cfcd22626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } } }
I20260811 22:18:49.393136 32086 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:49.393474 32087 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8e7d9c69c8f4810832df62cfcd22626. Latest consensus state: current_term: 2 leader_uuid: "b8e7d9c69c8f4810832df62cfcd22626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e7d9c69c8f4810832df62cfcd22626" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45407 } } }
I20260811 22:18:49.393570 32087 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:49.394027 32095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:49.397518 32095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:49.398165 32080 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:18:49.398372 32095 catalog_manager.cc:1259] Loaded cluster ID: 548c86d90b2045a292970cb2e3b78821
I20260811 22:18:49.398488 32095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:49.401178 32095 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:49.402011 32095 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Loaded TSK: 0
I20260811 22:18:49.403347 32095 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:18:49.515048 32082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:49.515584 32082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:49.515666 32082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:49.522956 32082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:49.523119 32082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:49.523231 32082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:18:49.532131 32082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:38355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=41365
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:49.533762 32082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:49.535288 32082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:49.548671 32125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.548694 32126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.548671 32128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:49.549285 32082 server_base.cc:1034] running on GCE node
I20260811 22:18:49.549844 32082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:49.550792 32082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:49.552000 32082 hybrid_clock.cc:648] HybridClock initialized: now 1786486729551981 us; error 41 us; skew 500 ppm
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:49.555997 32082 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:18:49.557350 32082 webserver.cc:460] Webserver started at http://127.17.198.1:41365/ using document root <none> and password file <none>
I20260811 22:18:49.558295 32082 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:49.558451 32082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:49.580304 32082 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:49.583710 32137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.585804 32082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:49.585997 32082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "9019b24563bb4cbab82b1294b1db7e76"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "11c526f84d9bfe04467499dd53e55595"
server_key_iv: "ef9642ae786ef88739bacb6a753d55ea"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:49.586858 32082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:49.606904 32082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:49.610754 32082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:49.611033 32082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:49.611821 32082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:49.613019 32082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:49.613090 32082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.613164 32082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:49.613190 32082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.631145 32082 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:38355
I20260811 22:18:49.631500 32269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:38355 every 8 connection(s)
I20260811 22:18:49.632692 32082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:49.649657 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32082
I20260811 22:18:49.650564 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:43083
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:45407
--webserver_port=42897
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:18:49.656109 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.635546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.1:50941)
0811 22:18:49.637325 (+ 1779us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.637369 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.638337 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.639631 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.639639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.640032 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:49.640605 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.640620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.642094 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.642098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.643569 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.643582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.644996 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.645000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.645193 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.646046 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:49.646078 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:49.648210 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:49.651493 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.651502 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.651520 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.651939 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.652499 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.652503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.652506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.652731 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.653816 (+ 1085us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:49.653823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:49.654171 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.654414 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.654868 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1563,"thread_start_us":147,"threads_started":1}
I20260811 22:18:49.658883 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.635424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:50941 (local address 127.17.198.62:45407)
0811 22:18:49.637026 (+ 1602us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.637046 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.637517 (+ 471us) server_negotiation.cc:407] Connection header received
0811 22:18:49.638834 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.638848 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.639220 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.639558 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.640799 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.641941 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.643734 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.644518 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.644705 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.648576 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.648585 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.648607 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.648651 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.651253 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.652135 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.652147 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.652156 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.652351 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.652904 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.652909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.652911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.653578 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.653708 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.657152 (+ 3444us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.657642 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1426,"thread_start_us":173,"threads_started":1}
I20260811 22:18:49.662062 32270 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:49.662402 32270 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.663157 32270 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:49.669373 32270 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:45407 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:18:49.669620 32270 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.669894 32270 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:49.671324 32270 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.671578 32270 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:49.672755 32270 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:18:49.811712 32274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:49.812021 32274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:49.812071 32274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:49.816603 32274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:49.816699 32274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:49.816769 32274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:18:49.823235 32274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:43083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=42897
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:49.824894 32274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:49.826047 32274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:49.837538 32284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.837509 32281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.837509 32280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:49.838227 32274 server_base.cc:1034] running on GCE node
I20260811 22:18:49.838680 32274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:49.839465 32274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:49.840643 32274 hybrid_clock.cc:648] HybridClock initialized: now 1786486729840607 us; error 50 us; skew 500 ppm
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:49.843778 32274 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:18:49.844866 32274 webserver.cc:460] Webserver started at http://127.17.198.2:42897/ using document root <none> and password file <none>
I20260811 22:18:49.845479 32274 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:49.845587 32274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:49.866966 32274 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:49.870749 32292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.872860 32274 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:49.873039 32274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "0c3fa8a893ff43aa94472fc257cbd6ff"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "4c559b5f984c8d06ff8de65b532d0066"
server_key_iv: "e5b0fa734fe0ac8186d93fa2534c2ffb"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:49.873932 32274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:49.926294 32274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:49.930637 32274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:49.930886 32274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:49.931718 32274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:49.932966 32274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:49.933081 32274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.933187 32274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:49.933241 32274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.946969 32274 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43083
I20260811 22:18:49.947032 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43083 every 8 connection(s)
I20260811 22:18:49.948639 32274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:49.952042 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32274
I20260811 22:18:49.952663 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.198.20:48629/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:41289
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:45407
--webserver_port=33381
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:18:49 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:49.973788 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.957060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:48767 (local address 127.17.198.62:45407)
0811 22:18:49.957280 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.957285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.957308 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:18:49.957368 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.957371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.957434 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.957553 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.959646 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.960500 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.962567 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.963002 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.963145 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.966682 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.966687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.966691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.966727 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.968843 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.971532 (+ 2689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.971538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.971540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.971664 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.972179 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.972183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.972186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.972462 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.972556 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.973418 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.973592 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:18:49.979035 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.952125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.2:48767)
0811 22:18:49.953939 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.954013 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.955244 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.958008 (+ 2764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.958025 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.958521 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:49.959386 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.959412 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.960715 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.960719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.962345 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.962362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.963374 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.963378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.963638 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.964473 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:49.964499 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:49.966363 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:49.969031 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.969040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.969062 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.971332 (+ 2270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.971798 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.971801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.971804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.972041 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.972634 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:49.972642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:49.973079 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.974543 (+ 1464us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.975108 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1508,"thread_start_us":195,"threads_started":1}
I20260811 22:18:49.981091 32424 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:49.981393 32424 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.982081 32424 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:49.984239 32424 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:45407 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:18:49.984397 32424 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.984611 32424 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:49.985939 32424 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.986179 32424 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:49.987327 32424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:18:50.119711 32428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:50.120028 32428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:50.120074 32428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:50.124714 32428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:50.124799 32428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:50.124963 32428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:18:50.131711 32428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.198.20:48629/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:41289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=33381
--tserver_master_addrs=127.17.198.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:50.133066 32428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:50.134347 32428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:50.148379 32440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:50.148416 32437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:50.148710 32438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:50.148562 32428 server_base.cc:1034] running on GCE node
I20260811 22:18:50.149437 32428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:50.150355 32428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:50.151621 32428 hybrid_clock.cc:648] HybridClock initialized: now 1786486730151593 us; error 48 us; skew 500 ppm
Aug 11 22:18:50 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:50.155759 32428 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:18:50.157334 32428 webserver.cc:460] Webserver started at http://127.17.198.3:33381/ using document root <none> and password file <none>
I20260811 22:18:50.158279 32428 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:50.158497 32428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:18:50 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:50.182727 32428 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:18:50.186111 32448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:50.187773 32428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:50.188031 32428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "ada79ffc4a2d4b2ba194ef313b4e212c"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-77v9"
server_key: "8a81759b2e9a7fc5abe7254ee09eabdf"
server_key_iv: "9852f1c29ac0867af17d68f159254d31"
server_key_version: "kuduclusterkey@0"
I20260811 22:18:50.189044 32428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:50.225605 32428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:50.230191 32428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:50.230461 32428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:50.231343 32428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:50.232702 32428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:50.232872 32428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:18:50.233031 32428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:50.233114 32428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:50.246825 32428 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:41289
I20260811 22:18:50.248581 32428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:50.249670 32581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:41289 every 8 connection(s)
I20260811 22:18:50.255383 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32428
I20260811 22:18:50.269657 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:50.256240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.17.198.62:45407)
0811 22:18:50.256592 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:50.256605 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:50.256802 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:18:50.257055 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:50.257058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:50.257112 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:50.257223 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:50.258090 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.258702 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.259651 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.259909 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.260034 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:50.264673 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:50.264678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:50.264680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:50.264708 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:50.266792 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:50.268008 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:50.268017 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:50.268019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:50.268115 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:50.268452 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:50.268456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:50.268458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:50.268664 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:50.269182 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:50.269273 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:18:50.269436 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":131,"threads_started":1}
Aug 11 22:18:50 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:50.278514 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:50.252441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:60781 (local address 127.17.198.62:45407)
0811 22:18:50.252593 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:50.252599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:50.263368 (+ 10769us) server_negotiation.cc:407] Connection header received
0811 22:18:50.264498 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:50.264502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:50.264567 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:50.264662 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:50.266511 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.267384 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.269296 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.269673 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.269770 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:50.273807 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:50.273811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:50.273814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:50.273896 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:50.275625 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:50.276415 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:50.276418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:50.276419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:50.276505 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:50.276983 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:50.276985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:50.276987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:50.277195 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:50.277280 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:50.278122 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 22:18:50.278288 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:18:50.279685 32584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:50.253035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:45407 (local address 127.17.198.3:60781)
0811 22:18:50.263172 (+ 10137us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:50.263244 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:50.264243 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:50.264993 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:50.265004 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:50.265449 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:50.266276 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:50.266301 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.267563 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.267566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:50.269117 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:50.269132 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:50.269994 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:50.269998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:50.270221 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:50.271471 (+ 1250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:50.271562 (+ 91us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:50.273544 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:50.275792 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:50.275800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:50.275823 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:50.276290 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:50.276602 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:50.276605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:50.276607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:50.276865 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:50.277321 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:50.277329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:50.277774 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 22:18:50.278013 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 22:18:50.278526 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9824,"thread_start_us":161,"threads_started":1}
I20260811 22:18:50.281244 32582 heartbeater.cc:347] Connected to a master server at 127.17.198.62:45407
I20260811 22:18:50.281546 32582 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:50.282235 32582 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:50.284456 32582 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:45407 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:18:50.284633 32582 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:50.284936 32582 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:50.286361 32582 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:50.286587 32582 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
W20260811 22:18:50.287510 32582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:18:50.673110 32270 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:50.673518 32270 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:50.675043 32038 ts_manager.cc:194] Registered new tserver with Master: 9019b24563bb4cbab82b1294b1db7e76 (127.17.198.1:38355)
I20260811 22:18:50.676928 32038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:50941
Aug 11 22:18:50 dist-test-slave-77v9 krb5kdc[30872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:50.987681 32424 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:50.988092 32424 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:50.989269 32033 ts_manager.cc:194] Registered new tserver with Master: 0c3fa8a893ff43aa94472fc257cbd6ff (127.17.198.2:43083)
I20260811 22:18:50.990196 32033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:48767
Aug 11 22:18:51 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:51.288141 32582 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:51.288889 32582 heartbeater.cc:511] Master 127.17.198.62:45407 requested a full tablet report, sending...
I20260811 22:18:51.290318 32033 ts_manager.cc:194] Registered new tserver with Master: ada79ffc4a2d4b2ba194ef313b4e212c (127.17.198.3:41289)
I20260811 22:18:51.291137 32033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:60781
I20260811 22:18:51.293109 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:18:51.306339 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.294994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.17.198.62:45407)
0811 22:18:51.295285 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.295290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.295560 (+ 270us) server_negotiation.cc:407] Connection header received
0811 22:18:51.295706 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.295709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.295755 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.295858 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:51.297007 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.297843 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.299242 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.299585 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.299725 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.301877 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:51.301882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:51.301885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:51.301917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:51.303936 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:51.304618 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:51.304621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:51.304623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:51.304724 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:51.305155 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:51.305160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:51.305162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:51.305434 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:51.305586 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.306005 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.306151 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260811 22:18:51.312163 32033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:51.318671 32033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:51.349108 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.337562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43083 (local address 127.0.0.1:53394)
0811 22:18:51.341829 (+ 4267us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.341846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.341978 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.342702 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.342707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.342728 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:51.342851 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.342857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.345098 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.345102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.346820 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.346832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.348452 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.348456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.348579 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.348636 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.348761 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.348934 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4192,"mutex_wait_us":30,"thread_start_us":188,"threads_started":1}
I20260811 22:18:51.349916 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.337822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.17.198.2:43083)
0811 22:18:51.339811 (+ 1989us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.339821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.342101 (+ 2280us) server_negotiation.cc:407] Connection header received
0811 22:18:51.342193 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.342198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.342393 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.342536 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:51.342993 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.344957 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.346978 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.348037 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.348153 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.348383 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.349587 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.349717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1846,"thread_start_us":150,"threads_started":1}
I20260811 22:18:51.353484 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.335756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:38355 (local address 127.0.0.1:46118)
0811 22:18:51.340204 (+ 4448us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.340290 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.340447 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.343019 (+ 2572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.343022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.343042 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:51.343223 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.343230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.344911 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.344915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.347501 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.347511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.353014 (+ 5503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.353018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.353128 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.353159 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.353237 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.353326 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4213,"mutex_wait_us":1046,"thread_start_us":274,"threads_started":1}
I20260811 22:18:51.354462 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.336930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.17.198.1:38355)
0811 22:18:51.340855 (+ 3925us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.340863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.340892 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:18:51.340978 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.340985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.341289 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.341497 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:51.343377 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.344706 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.348402 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.349454 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.349583 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.349734 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.354115 (+ 4381us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.354252 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3771,"thread_start_us":141,"threads_started":1}
I20260811 22:18:51.354457 32340 tablet_service.cc:1467] Processing CreateTablet for tablet 859a006325384cb2b1d9df970613ffdf (DEFAULT_TABLE table=test-table [id=b676fb53e7424c099659cc5a7b4be06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:51.355990 32340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 859a006325384cb2b1d9df970613ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:51.356666 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.338434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:41289 (local address 127.0.0.1:43126)
0811 22:18:51.345726 (+ 7292us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.345743 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.345871 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.348049 (+ 2178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.348053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.348078 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:51.348243 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.348251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.351512 (+ 3261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.351517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.352850 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.352865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.356156 (+ 3291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.356160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.356280 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.356308 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.356391 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.356477 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7214,"thread_start_us":249,"threads_started":1}
I20260811 22:18:51.358698 32615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.339026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.17.198.3:41289)
0811 22:18:51.346138 (+ 7112us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.346146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.346173 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:18:51.346252 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.346258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.347633 (+ 1375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.347832 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:51.349957 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.351348 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.354771 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.355776 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.355906 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.356074 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.358355 (+ 2281us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.358493 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6991,"thread_start_us":224,"threads_started":1}
I20260811 22:18:51.360849 32185 tablet_service.cc:1467] Processing CreateTablet for tablet 859a006325384cb2b1d9df970613ffdf (DEFAULT_TABLE table=test-table [id=b676fb53e7424c099659cc5a7b4be06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:51.362224 32185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 859a006325384cb2b1d9df970613ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:51.362353 32483 tablet_service.cc:1467] Processing CreateTablet for tablet 859a006325384cb2b1d9df970613ffdf (DEFAULT_TABLE table=test-table [id=b676fb53e7424c099659cc5a7b4be06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:51.363766 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 859a006325384cb2b1d9df970613ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:51.369302 32616 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Bootstrap starting.
I20260811 22:18:51.375924 32618 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Bootstrap starting.
I20260811 22:18:51.379658 32618 tablet_bootstrap.cc:654] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:51.381436 32618 log.cc:824] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:51.382172 32616 tablet_bootstrap.cc:654] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:51.383558 32616 log.cc:824] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:51.384459 32617 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: Bootstrap starting.
I20260811 22:18:51.385320 32618 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: No bootstrap required, opened a new log
I20260811 22:18:51.385546 32618 ts_tablet_manager.cc:1397] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 22:18:51.387634 32617 tablet_bootstrap.cc:654] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:51.389024 32617 log.cc:824] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:51.393203 32617 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: No bootstrap required, opened a new log
I20260811 22:18:51.392724 32618 raft_consensus.cc:348] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.393299 32618 raft_consensus.cc:374] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:51.393365 32618 raft_consensus.cc:729] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada79ffc4a2d4b2ba194ef313b4e212c, State: Initialized, Role: FOLLOWER
I20260811 22:18:51.393460 32617 ts_tablet_manager.cc:1397] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:18:51.394143 32618 consensus_queue.cc:260] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.396189 32618 ts_tablet_manager.cc:1428] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 22:18:51.396713 32582 heartbeater.cc:503] Master 127.17.198.62:45407 was elected leader, sending a full tablet report...
I20260811 22:18:51.400835 32617 raft_consensus.cc:348] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.401315 32617 raft_consensus.cc:374] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:51.401440 32617 raft_consensus.cc:729] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9019b24563bb4cbab82b1294b1db7e76, State: Initialized, Role: FOLLOWER
I20260811 22:18:51.404225 32616 tablet_bootstrap.cc:492] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: No bootstrap required, opened a new log
I20260811 22:18:51.404469 32616 ts_tablet_manager.cc:1397] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.006s
I20260811 22:18:51.404682 32622 raft_consensus.cc:482] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:51.404932 32622 raft_consensus.cc:504] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.408401 32622 leader_election.cc:288] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9019b24563bb4cbab82b1294b1db7e76 (127.17.198.1:38355), 0c3fa8a893ff43aa94472fc257cbd6ff (127.17.198.2:43083)
I20260811 22:18:51.409225 32616 raft_consensus.cc:348] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.409678 32616 raft_consensus.cc:374] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:51.409770 32616 raft_consensus.cc:729] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c3fa8a893ff43aa94472fc257cbd6ff, State: Initialized, Role: FOLLOWER
I20260811 22:18:51.411543 32616 consensus_queue.cc:260] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.412940 32616 ts_tablet_manager.cc:1428] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260811 22:18:51.413192 32617 consensus_queue.cc:260] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.414345 32617 ts_tablet_manager.cc:1428] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260811 22:18:51.414799 32424 heartbeater.cc:503] Master 127.17.198.62:45407 was elected leader, sending a full tablet report...
I20260811 22:18:51.417400 32270 heartbeater.cc:503] Master 127.17.198.62:45407 was elected leader, sending a full tablet report...
I20260811 22:18:51.423044 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.406557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:38355 (local address 127.17.198.3:49387)
0811 22:18:51.417533 (+ 10976us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.417554 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.418236 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.418679 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.418686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.418710 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:51.418904 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.418914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.420098 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.420102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.421625 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.421637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.422541 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.422545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.422673 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.422706 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.422798 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.422874 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10875,"thread_start_us":159,"threads_started":1}
I20260811 22:18:51.423050 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.407037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:49387 (local address 127.17.198.1:38355)
0811 22:18:51.407277 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.407283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.417653 (+ 10370us) server_negotiation.cc:407] Connection header received
0811 22:18:51.418400 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.418404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.418454 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.418510 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:51.419053 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.419932 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.421793 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.422429 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.422545 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.422598 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.422837 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.422918 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:18:51.424011 32218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "859a006325384cb2b1d9df970613ffdf" candidate_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9019b24563bb4cbab82b1294b1db7e76" is_pre_election: true
I20260811 22:18:51.424392 32218 raft_consensus.cc:2454] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada79ffc4a2d4b2ba194ef313b4e212c in term 0.
I20260811 22:18:51.425264 32451 leader_election.cc:302] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9019b24563bb4cbab82b1294b1db7e76, ada79ffc4a2d4b2ba194ef313b4e212c; no voters:
I20260811 22:18:51.425877 32625 raft_consensus.cc:2781] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:51.426084 32625 raft_consensus.cc:482] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:51.426210 32625 raft_consensus.cc:3037] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:51.426869 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.408292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47563 (local address 127.17.198.2:43083)
0811 22:18:51.408468 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.408473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.418510 (+ 10037us) server_negotiation.cc:407] Connection header received
0811 22:18:51.418562 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.418565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.418622 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.418699 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:51.421269 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.422054 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.425519 (+ 3465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.426117 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.426216 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.426267 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.426646 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.426729 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:18:51.427544 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.407655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43083 (local address 127.17.198.3:47563)
0811 22:18:51.417864 (+ 10209us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.417880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.417999 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.419119 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.419122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.419144 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:51.419383 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.419392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.422202 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.422206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.423477 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:51.425352 (+ 1875us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.426379 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.426383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:51.426507 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.426530 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.427298 (+ 768us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.427402 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10137,"thread_start_us":189,"threads_started":1}
I20260811 22:18:51.432289 32625 raft_consensus.cc:504] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.432989 32625 leader_election.cc:288] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [CANDIDATE]: Term 1 election: Requested vote from peers 9019b24563bb4cbab82b1294b1db7e76 (127.17.198.1:38355), 0c3fa8a893ff43aa94472fc257cbd6ff (127.17.198.2:43083)
I20260811 22:18:51.436050 32355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "859a006325384cb2b1d9df970613ffdf" candidate_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" is_pre_election: true
I20260811 22:18:51.436604 32355 raft_consensus.cc:2454] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada79ffc4a2d4b2ba194ef313b4e212c in term 0.
I20260811 22:18:51.437449 32353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "859a006325384cb2b1d9df970613ffdf" candidate_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff"
I20260811 22:18:51.437712 32353 raft_consensus.cc:3037] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:51.438004 32218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "859a006325384cb2b1d9df970613ffdf" candidate_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9019b24563bb4cbab82b1294b1db7e76"
I20260811 22:18:51.438174 32218 raft_consensus.cc:3037] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:51.440026 32218 raft_consensus.cc:2454] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada79ffc4a2d4b2ba194ef313b4e212c in term 1.
I20260811 22:18:51.440456 32451 leader_election.cc:302] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9019b24563bb4cbab82b1294b1db7e76, ada79ffc4a2d4b2ba194ef313b4e212c; no voters:
I20260811 22:18:51.440675 32625 raft_consensus.cc:2781] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:51.440846 32353 raft_consensus.cc:2454] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada79ffc4a2d4b2ba194ef313b4e212c in term 1.
I20260811 22:18:51.442710 32625 raft_consensus.cc:686] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 1 LEADER]: Becoming Leader. State: Replica: ada79ffc4a2d4b2ba194ef313b4e212c, State: Running, Role: LEADER
I20260811 22:18:51.443298 32625 consensus_queue.cc:237] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } }
I20260811 22:18:51.448968 32038 catalog_manager.cc:5495] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c reported cstate change: term changed from 0 to 1, leader changed from <none> to ada79ffc4a2d4b2ba194ef313b4e212c (127.17.198.3). New cstate: current_term: 1 leader_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada79ffc4a2d4b2ba194ef313b4e212c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 41289 } health_report { overall_health: HEALTHY } } }
W20260811 22:18:51.451538 32425 tablet.cc:2369] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:18:51 dist-test-slave-77v9 krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486730, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for HTTP/127.17.198.20@KRBTEST.COM
I20260811 22:18:51.472704 32615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.466512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.17.198.3:41289)
0811 22:18:51.466673 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.466679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.466698 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:18:51.466818 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.466823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.466909 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.467048 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:51.467548 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.468376 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.469907 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:51.470321 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:51.470448 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:51.470984 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:51.471667 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:51.471817 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.472419 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.472499 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:18:51.485739 32218 raft_consensus.cc:1261] T 859a006325384cb2b1d9df970613ffdf P 9019b24563bb4cbab82b1294b1db7e76 [term 1 FOLLOWER]: Refusing update from remote peer ada79ffc4a2d4b2ba194ef313b4e212c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:51.485852 32353 raft_consensus.cc:1261] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 1 FOLLOWER]: Refusing update from remote peer ada79ffc4a2d4b2ba194ef313b4e212c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:51.486778 32622 consensus_queue.cc:1035] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9019b24563bb4cbab82b1294b1db7e76" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:51.488037 32622 consensus_queue.cc:1035] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c3fa8a893ff43aa94472fc257cbd6ff" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:51.501928 32635 mvcc.cc:205] Tried to move back new op lower bound from 7317449652154019840 to 7317449651996143616. Current Snapshot: MvccSnapshot[applied={T|T < 7317449652154019840}]
I20260811 22:18:51.503961 32636 mvcc.cc:205] Tried to move back new op lower bound from 7317449652154019840 to 7317449651996143616. Current Snapshot: MvccSnapshot[applied={T|T < 7317449652154019840}]
I20260811 22:18:51.504837 32638 mvcc.cc:205] Tried to move back new op lower bound from 7317449652154019840 to 7317449651996143616. Current Snapshot: MvccSnapshot[applied={T|T < 7317449652154019840}]
I20260811 22:18:51.515723 32038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:51.515944 32038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:51.518908 32038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8e7d9c69c8f4810832df62cfcd22626: Sending DeleteTablet for 3 replicas of tablet 859a006325384cb2b1d9df970613ffdf
I20260811 22:18:51.520169 32340 tablet_service.cc:1514] Processing DeleteTablet for tablet 859a006325384cb2b1d9df970613ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:51 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:53394
I20260811 22:18:51.520967 32483 tablet_service.cc:1514] Processing DeleteTablet for tablet 859a006325384cb2b1d9df970613ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:51 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:43126
I20260811 22:18:51.521999 32185 tablet_service.cc:1514] Processing DeleteTablet for tablet 859a006325384cb2b1d9df970613ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:51 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:46118
I20260811 22:18:51.523249 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32082
I20260811 22:18:51.532421 32647 tablet_replica.cc:330] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: stopping tablet replica
I20260811 22:18:51.533084 32648 tablet_replica.cc:330] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: stopping tablet replica
I20260811 22:18:51.533543 32648 raft_consensus.cc:2229] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:51.534057 32648 raft_consensus.cc:2258] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:51.534400 32647 raft_consensus.cc:2229] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:51.534772 32647 raft_consensus.cc:2258] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:51.536804 32648 ts_tablet_manager.cc:1905] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:51.542841 32647 ts_tablet_manager.cc:1905] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:51.546085 32647 ts_tablet_manager.cc:1918] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:51.546232 32647 log.cc:1196] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/859a006325384cb2b1d9df970613ffdf
I20260811 22:18:51.546643 32647 ts_tablet_manager.cc:1939] T 859a006325384cb2b1d9df970613ffdf P 0c3fa8a893ff43aa94472fc257cbd6ff: Deleting consensus metadata
I20260811 22:18:51.547192 32648 ts_tablet_manager.cc:1918] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:51.547314 32648 log.cc:1196] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/859a006325384cb2b1d9df970613ffdf
I20260811 22:18:51.547683 32648 ts_tablet_manager.cc:1939] T 859a006325384cb2b1d9df970613ffdf P ada79ffc4a2d4b2ba194ef313b4e212c: Deleting consensus metadata
I20260811 22:18:51.550001 32022 catalog_manager.cc:4841] TS ada79ffc4a2d4b2ba194ef313b4e212c (127.17.198.3:41289): tablet 859a006325384cb2b1d9df970613ffdf (table test-table [id=b676fb53e7424c099659cc5a7b4be06e]) successfully deleted
I20260811 22:18:51.550575 32022 catalog_manager.cc:4841] TS 0c3fa8a893ff43aa94472fc257cbd6ff (127.17.198.2:43083): tablet 859a006325384cb2b1d9df970613ffdf (table test-table [id=b676fb53e7424c099659cc5a7b4be06e]) successfully deleted
W20260811 22:18:51.551841 32022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:38355 (error 108)
I20260811 22:18:51.553231 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.552884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:38355 (local address 127.0.0.1:46124)
0811 22:18:51.553074 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.553142 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:38355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260811 22:18:51.553400 32022 catalog_manager.cc:4584] TS 9019b24563bb4cbab82b1294b1db7e76 (127.17.198.1:38355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 859a006325384cb2b1d9df970613ffdf: Network error: Client connection negotiation failed: client connection to 127.17.198.1:38355: connect: Connection refused (error 111)
I20260811 22:18:51.561058 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32274
I20260811 22:18:51.571980 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32428
I20260811 22:18:51.584977 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32000
I20260811 22:18:51.599637 18200 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:18:51.945856 18200 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:18:51.946018 18200 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:18:52.293614 18200 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:18:52.293 UTC [30910] LOG: received smart shutdown request
2026-08-11 22:18:52.299 UTC [30910] LOG: background worker "logical replication launcher" (PID 30919) exited with exit code 1
2026-08-11 22:18:52.299 UTC [30914] LOG: shutting down
2026-08-11 22:18:52.533 UTC [30910] LOG: database system is shut down
2026-08-11T22:18:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56252 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:52 dist-test-slave-77v9 krb5kdc[32661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:52 dist-test-slave-77v9 krb5kdc[32661](info): set up 2 sockets
Aug 11 22:18:52 dist-test-slave-77v9 krb5kdc[32661](info): commencing operation
krb5kdc: starting...
W20260811 22:18:53.869719 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:18:53 dist-test-slave-77v9 krb5kdc[32661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:18:54.017851 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37175
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:54.142035 32676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:54.142369 32676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:54.142416 32676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:54.146636 32676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:54.146713 32676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:54.146733 32676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:54.146756 32676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:54.146772 32676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:54.153833 32676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:54.155160 32676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:54.156339 32676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:54.166270 32686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.166270 32684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:54.166555 32676 server_base.cc:1034] running on GCE node
W20260811 22:18:54.166711 32683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:54.167371 32676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:54.168797 32676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:54.170056 32676 hybrid_clock.cc:648] HybridClock initialized: now 1786486734170027 us; error 49 us; skew 500 ppm
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:54.173525 32676 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:18:54.174686 32676 webserver.cc:460] Webserver started at http://127.17.198.62:41159/ using document root <none> and password file <none>
I20260811 22:18:54.175304 32676 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:54.175406 32676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:54.175658 32676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:54.178043 32676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "72df51056a454dd9a0c15c986242661d"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "003f585ffc3bf1241cf350a5afbdd019"
server_key_iv: "c747285de76fdf092f9c6c6e755ae144"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.178678 32676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "72df51056a454dd9a0c15c986242661d"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "003f585ffc3bf1241cf350a5afbdd019"
server_key_iv: "c747285de76fdf092f9c6c6e755ae144"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.183033 32676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:18:54.186074 32694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:54.187527 32676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:18:54.187683 32676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "72df51056a454dd9a0c15c986242661d"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "003f585ffc3bf1241cf350a5afbdd019"
server_key_iv: "c747285de76fdf092f9c6c6e755ae144"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.187815 32676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:54.203729 32676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:54.207669 32676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:54.207945 32676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:54.216109 32676 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37175
I20260811 22:18:54.216144 32755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37175 every 8 connection(s)
I20260811 22:18:54.217236 32676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:54.220398 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:54.224733 32756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.226831 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32676
I20260811 22:18:54.227106 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:18:54.227391 18200 external_mini_cluster.cc:1428] Setting key 2a157275d611db0e36d97a8f8597fa33
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.61@KRBTEST.COM" created.
I20260811 22:18:54.242749 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.227480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37175 (local address 127.0.0.1:33364)
0811 22:18:54.228763 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.228861 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.229865 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.230626 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.230632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.231128 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.231396 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.231410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.232594 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.232598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.233443 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.233453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.234125 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.234129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.234236 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.235019 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.235037 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.236792 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.239116 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.239128 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.239139 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.239403 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.239779 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.239782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.239784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.240047 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.240806 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.240911 (+ 105us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.241065 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.241244 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.241689 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":113,"thread_start_us":205,"threads_started":1}
I20260811 22:18:54.242749 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.227480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.17.198.62:37175)
0811 22:18:54.228186 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.228206 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.228996 (+ 790us) server_negotiation.cc:407] Connection header received
0811 22:18:54.230214 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.230219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.230424 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.230640 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.231556 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.232450 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.233567 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.233955 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.234134 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.237019 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.237026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.237038 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.237062 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.238944 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.239527 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.239532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.239533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.239699 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.240180 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.240183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.240184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.240679 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.240872 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.241326 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.241605 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"spinlock_wait_cycles":112000,"thread_start_us":215,"threads_started":1}
I20260811 22:18:54.245803 32756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.246726 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.246428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40100)
0811 22:18:54.246574 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.246658 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:18:54.247020 32697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
I20260811 22:18:54.248021 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.247756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40114)
0811 22:18:54.247921 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.247967 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:18:54.248252 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:43435: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
I20260811 22:18:54.261576 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:43435
--webserver_interface=127.17.198.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:18:54.297544 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } attempt: 1
I20260811 22:18:54.298552 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.298180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40130)
0811 22:18:54.298392 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.298462 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:18:54.299482 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.299168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40132)
0811 22:18:54.299314 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.299407 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:18:54.299706 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:43435: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
I20260811 22:18:54.367924 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } attempt: 2
I20260811 22:18:54.368973 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.368593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40144)
0811 22:18:54.368764 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.368889 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:18:54.369757 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.369519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40152)
0811 22:18:54.369640 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.369696 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:18:54.369930 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:43435: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
W20260811 22:18:54.385191 32764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:54.385556 32764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:54.385648 32764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:54.390002 32764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:54.390112 32764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:54.390152 32764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:54.390210 32764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:54.390249 32764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:54.396373 32764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:43435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.198.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:54.397782 32764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:54.399016 32764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:54.408910 305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.408959 301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.408915 303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:54.409163 32764 server_base.cc:1034] running on GCE node
I20260811 22:18:54.409888 32764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:54.411155 32764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:54.412366 32764 hybrid_clock.cc:648] HybridClock initialized: now 1786486734412349 us; error 38 us; skew 500 ppm
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:54.415982 32764 init.cc:377] Logged in from keytab as kudu/127.17.198.61@KRBTEST.COM (short username kudu)
I20260811 22:18:54.417153 32764 webserver.cc:460] Webserver started at http://127.17.198.61:41067/ using document root <none> and password file <none>
I20260811 22:18:54.417815 32764 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:54.417915 32764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:54.418165 32764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:54.420670 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data/instance:
uuid: "eaff209ba6864a35aa5a08e56429d545"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "89d201bb1f9e755e8648b59689cdb273"
server_key_iv: "4c5235bbec9114b06507626ea5861c5b"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.421393 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal/instance:
uuid: "eaff209ba6864a35aa5a08e56429d545"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "89d201bb1f9e755e8648b59689cdb273"
server_key_iv: "4c5235bbec9114b06507626ea5861c5b"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.425782 32764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:18:54.428740 314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:54.430331 32764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:18:54.430516 32764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "eaff209ba6864a35aa5a08e56429d545"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "89d201bb1f9e755e8648b59689cdb273"
server_key_iv: "4c5235bbec9114b06507626ea5861c5b"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.430657 32764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:54.459895 32764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:54.461105 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } attempt: 3
I20260811 22:18:54.462075 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.461699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40164)
0811 22:18:54.461873 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.461938 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 22:18:54.463025 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.462784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40178)
0811 22:18:54.462889 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.462942 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:18:54.463198 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:43435: Network error: Client connection negotiation failed: client connection to 127.17.198.61:43435: connect: Connection refused (error 111)
I20260811 22:18:54.464311 32764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:54.464529 32764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:54.472497 32764 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.61:43435
I20260811 22:18:54.472550 380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.61:43435 every 8 connection(s)
I20260811 22:18:54.473719 32764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
I20260811 22:18:54.477128 381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:54.481236 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 32764
I20260811 22:18:54.481413 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-1/wal/instance
I20260811 22:18:54.481712 18200 external_mini_cluster.cc:1428] Setting key a3f82b9135b45f74ac629fbca3e79859
I20260811 22:18:54.482244 381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.198.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.60@KRBTEST.COM" created.
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:54.501634 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.484515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.17.198.62:37175)
0811 22:18:54.484707 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.484713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.485853 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 22:18:54.487125 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.487133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.487226 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.487358 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.489002 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.490036 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.491450 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.491704 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.491827 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.496624 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.496628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.496630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.496662 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.498646 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.499439 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.499445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.499450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.499563 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.500063 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.500070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.500074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.500385 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.500497 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.501256 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.501429 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:18:54.502581 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.485029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37175 (local address 127.0.0.1:33380)
0811 22:18:54.485638 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.485708 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.486823 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.487561 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.487569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.488139 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.488704 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.488719 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.490220 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.490223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.491288 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.491299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.491970 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.491975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.492148 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.492938 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.492956 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.496378 (+ 3422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.498907 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.498914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.498931 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.499291 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.499678 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.499681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.499683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.499938 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.500525 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.500532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.500910 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.501138 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.501486 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":180,"threads_started":1}
I20260811 22:18:54.504364 381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:18:54.515269 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.505079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40186)
0811 22:18:54.505571 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.505588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.505692 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.506113 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.506116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.506339 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.506428 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.506434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.507472 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.507477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.508055 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.508064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.508711 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.508715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.508913 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.509444 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.509457 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.510890 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.513403 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.513408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.513410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.513643 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.513985 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.513990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.513993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.514082 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.514628 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.514631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.514730 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.514851 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.515036 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441}
I20260811 22:18:54.515295 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.505205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.17.198.61:43435)
0811 22:18:54.505572 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.505587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.505644 (+ 57us) server_negotiation.cc:407] Connection header received
0811 22:18:54.505798 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.505807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.505993 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.506140 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.506541 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.507342 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.508177 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.508588 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.508709 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.511070 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.511083 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.511103 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.511131 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.513227 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.513789 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.513795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.513796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.513884 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.514183 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.514187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.514188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.514508 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.514662 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.514964 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.515116 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
I20260811 22:18:54.517086 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:35855
--webserver_interface=127.17.198.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:18:54.518031 381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.518990 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.518614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42210)
0811 22:18:54.518775 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.518880 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:18:54.519308 317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
I20260811 22:18:54.520376 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.520112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42212)
0811 22:18:54.520218 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.520265 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:18:54.520677 381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
I20260811 22:18:54.570010 381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 1
I20260811 22:18:54.571165 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.570806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42216)
0811 22:18:54.571011 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.571080 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:18:54.572023 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.571772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42224)
0811 22:18:54.571883 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.571952 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:18:54.572165 381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
I20260811 22:18:54.606401 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } attempt: 4
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:18:54.618251 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.607175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40190)
0811 22:18:54.607442 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.607460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.607614 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.607939 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.607942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.608165 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.608290 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.608297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.609669 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.609673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.610896 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.610908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.611493 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.611496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.611615 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.612222 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.612236 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.613930 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.616383 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.616387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.616390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.616642 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.617068 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.617070 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.617072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.617176 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.617699 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.617701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.617802 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.617979 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.618119 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 22:18:54.618399 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.607330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.17.198.61:43435)
0811 22:18:54.607474 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.607481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.607541 (+ 60us) server_negotiation.cc:407] Connection header received
0811 22:18:54.607730 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.607733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.607794 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.607913 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.608421 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.609527 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.611044 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.611330 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.611468 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.614141 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.614148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.614154 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.614196 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.616192 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.616767 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.616771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.616773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.616928 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.617284 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.617288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.617290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.617588 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.617705 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.618068 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.618226 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:18:54.619412 32756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.620287 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.620006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42230)
0811 22:18:54.620161 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.620218 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:18:54.621076 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.620771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42232)
0811 22:18:54.620973 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.621020 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 22:18:54.621281 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
I20260811 22:18:54.640362 381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 2
I20260811 22:18:54.641290 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.640983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42246)
0811 22:18:54.641139 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.641204 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:18:54.642050 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.641830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42248)
0811 22:18:54.641938 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.641984 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:18:54.642230 381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
W20260811 22:18:54.653257 388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:54.653652 388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:54.653744 388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:54.658247 388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:54.658362 388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:54.658409 388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:54.658466 388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:54.658519 388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:54.664410 388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37175,127.17.198.61:43435,127.17.198.60:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.198.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:54.665791 388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:54.666946 388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:54.676383 397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.676365 394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.676367 395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:54.676781 388 server_base.cc:1034] running on GCE node
I20260811 22:18:54.677498 388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:54.678748 388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:54.679948 388 hybrid_clock.cc:648] HybridClock initialized: now 1786486734679918 us; error 46 us; skew 500 ppm
I20260811 22:18:54.680460 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 1
I20260811 22:18:54.681365 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.681074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42262)
0811 22:18:54.681215 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.681279 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:18:54.682103 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.681891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42266)
0811 22:18:54.682000 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.682051 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:18:54.682251 32756 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:54.683537 388 init.cc:377] Logged in from keytab as kudu/127.17.198.60@KRBTEST.COM (short username kudu)
I20260811 22:18:54.684625 388 webserver.cc:460] Webserver started at http://127.17.198.60:34789/ using document root <none> and password file <none>
I20260811 22:18:54.685397 388 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:54.685503 388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:54.685770 388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:54.688298 388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data/instance:
uuid: "375e329f3f58472287b8b3407b8ef015"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "21e0961c2d4f338ff6e00a46524b66a7"
server_key_iv: "b2979cc5ead9667d5bb9cf6e1df0b4d3"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.689026 388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal/instance:
uuid: "375e329f3f58472287b8b3407b8ef015"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "21e0961c2d4f338ff6e00a46524b66a7"
server_key_iv: "b2979cc5ead9667d5bb9cf6e1df0b4d3"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.693578 388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:54.696539 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:54.698066 388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:54.698252 388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "375e329f3f58472287b8b3407b8ef015"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-77v9"
server_key: "21e0961c2d4f338ff6e00a46524b66a7"
server_key_iv: "b2979cc5ead9667d5bb9cf6e1df0b4d3"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.698433 388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:54.733444 381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 3
I20260811 22:18:54.734236 388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:54.734423 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.734069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42280)
0811 22:18:54.734240 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.734309 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 22:18:54.735311 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.735029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42288)
0811 22:18:54.735178 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.735232 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 22:18:54.735512 381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:35855: Network error: Client connection negotiation failed: client connection to 127.17.198.60:35855: connect: Connection refused (error 111)
I20260811 22:18:54.738781 388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:54.738981 388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:54.748368 388 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.60:35855
I20260811 22:18:54.748400 467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.60:35855 every 8 connection(s)
I20260811 22:18:54.749531 388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
I20260811 22:18:54.749430 32756 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 2
I20260811 22:18:54.752918 468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:54.756927 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 388
I20260811 22:18:54.757103 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486470476166-18200-0/minicluster-data/master-2/wal/instance
I20260811 22:18:54.757480 18200 external_mini_cluster.cc:1428] Setting key 0bcabc36076519a5dcca206c78614c8d
I20260811 22:18:54.758611 468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:18:54.765544 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.750201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42298)
0811 22:18:54.750398 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.750416 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.750524 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.753062 (+ 2538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.753067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.753339 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.753497 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.753509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.755010 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.755019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.755948 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.755966 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.757232 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.757236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.757356 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.758017 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.758038 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.759946 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.763019 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.763022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.763024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.763376 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.763880 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.763884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.763886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.764004 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.764952 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.764955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.765065 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.765189 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.765352 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":3}
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:18:54.767671 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.750944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.17.198.60:35855)
0811 22:18:54.751387 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.751413 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.751462 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:18:54.752228 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.752246 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.752598 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.753051 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.753682 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.754828 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.756128 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.757023 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.757743 (+ 720us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.760214 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.760223 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.760259 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.760300 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.762887 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.763500 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.763510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.763518 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.763698 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.764118 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.764121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.764123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.764715 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.764890 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.765373 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.766215 (+ 842us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":155,"threads_started":1}
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:54.772362 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.758621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.17.198.60:35855)
0811 22:18:54.759073 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.759081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.759345 (+ 264us) server_negotiation.cc:407] Connection header received
0811 22:18:54.759524 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.759530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.759592 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.759730 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.760978 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.761879 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.763236 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.763657 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.763788 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.767011 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.767015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.767018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.767046 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.769746 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.770405 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.770409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.770411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.770512 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.770891 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.770897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.770899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.771170 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.771390 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.772016 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.772162 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":163,"threads_started":1}
I20260811 22:18:54.773778 32756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: Bootstrap starting.
I20260811 22:18:54.776026 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.761765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.17.198.62:37175)
0811 22:18:54.761927 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.761933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.763365 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 22:18:54.763618 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.763623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.763683 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.763792 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.764870 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.765805 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.767257 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.767586 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.767729 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.771536 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.771541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.771544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.771582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.773525 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.774239 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.774246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.774251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.774363 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.774649 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.774652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.774654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.775099 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.775227 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.775730 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.775855 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":67}
I20260811 22:18:54.776098 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.761366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37175 (local address 127.0.0.1:33390)
0811 22:18:54.763188 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.763264 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.763456 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.763970 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.763979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.764532 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.764684 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.764693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.765945 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.765948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.767106 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.767119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.767741 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.767745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.767890 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.768416 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.768439 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.771389 (+ 2950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.773652 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.773665 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.773671 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.774108 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.774475 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.774477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.774478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.774549 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.775244 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.775254 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.775473 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.775661 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.775900 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":124,"threads_started":1}
I20260811 22:18:54.777170 32756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:54.777550 468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.778242 32756 log.cc:824] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:54.780793 32756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: No bootstrap required, opened a new log
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:18:54.786761 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.774111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.17.198.62:37175)
0811 22:18:54.774446 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.774470 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.774485 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:18:54.774523 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.774526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.774576 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.774723 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.775487 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.776588 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.777891 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.778589 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.779145 (+ 556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.781761 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.781765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.781768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.781798 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.783704 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.784333 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.784340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.784345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.785092 (+ 747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.785458 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.785464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.785471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.785757 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.785986 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.786208 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.786357 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
I20260811 22:18:54.787197 32756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.787539 32756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:54.787611 32756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72df51056a454dd9a0c15c986242661d, State: Initialized, Role: FOLLOWER
I20260811 22:18:54.788198 32756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.788465 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.778153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:43435 (local address 127.0.0.1:40194)
0811 22:18:54.778276 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.778292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.778391 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.778703 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.778709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.778906 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.779032 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.779043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.780482 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.780488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.781443 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.781455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.782089 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.782094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.782196 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.782639 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.782655 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.784286 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.786607 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.786611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.786614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.786915 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.787261 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.787266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.787268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.787380 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.787877 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.787879 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.787982 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.788096 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.788277 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 22:18:54.788523 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.778275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.17.198.61:43435)
0811 22:18:54.778429 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.778434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.778451 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:18:54.778499 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.778503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.778569 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.778671 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.779545 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.780317 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.781587 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.781931 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.782074 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.784472 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.784479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.784485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.784525 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.786465 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.787037 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.787041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.787043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.787155 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.787476 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.787479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.787481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.787725 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.787826 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.788189 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.788347 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:18:54.788981 476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:54.789124 476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:54.789440 468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:18:54.790035 32756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
W20260811 22:18:54.797299 491 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:18:54.797394 491 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:18:54.796514 32676 master_runner.cc:186] Error getting master registration for 127.17.198.61:43435: OK, instance_id { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" instance_seqno: 1786486734470215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:18:54.801945 32676 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:18:54.802111 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.790130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.17.198.60:35855)
0811 22:18:54.790234 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.790238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.790259 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:18:54.790291 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.790293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.790331 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.790402 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.790892 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.791605 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.792690 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.793090 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.793161 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.798368 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.798372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.798374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.798402 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.800249 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.800726 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.800730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.800732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.800840 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.801108 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.801110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.801112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.801380 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.801457 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.801771 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.801921 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:18:54.802639 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.789973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42306)
0811 22:18:54.790083 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.790096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.790168 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.790451 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.790455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.790651 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.790749 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.790755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.791776 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.791783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.792570 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.792580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.793354 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.793358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.793447 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.794050 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.794066 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.798203 (+ 4137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.800375 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.800379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.800381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.800622 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.800941 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.800944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.800946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.801025 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.801516 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.801519 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.801605 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.802306 (+ 701us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.802444 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 22:18:54.802780 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.788568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.17.198.61:43435)
0811 22:18:54.788727 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.788731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.789672 (+ 941us) server_negotiation.cc:407] Connection header received
0811 22:18:54.789808 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.789811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.789864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.789952 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.790978 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.791888 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.793019 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.793353 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.793447 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.797586 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.797590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.797594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.797624 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.799810 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.800454 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.800458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.800461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.800603 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.801030 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.801034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.801036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.801317 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.801593 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.802428 (+ 835us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.802577 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":134,"threads_started":1}
I20260811 22:18:54.806712 468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: Bootstrap starting.
I20260811 22:18:54.807039 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:54.810451 468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:54.811573 468 log.cc:824] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:54.814458 468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: No bootstrap required, opened a new log
I20260811 22:18:54.819379 468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.819727 468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:54.819792 468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375e329f3f58472287b8b3407b8ef015, State: Initialized, Role: FOLLOWER
I20260811 22:18:54.820427 468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.823050 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.810483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.17.198.61:43435)
0811 22:18:54.810828 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.810834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.810852 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:18:54.810905 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.810908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.810960 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.811047 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.812393 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.813168 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.814110 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.814524 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.814666 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.817026 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.817031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.817034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.817069 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.819911 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.820524 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.820530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.820952 (+ 422us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.821085 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.821782 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.821785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.821788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.822091 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.822224 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.822694 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.822839 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260811 22:18:54.822149 468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:54.825682 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.810704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.17.198.60:35855)
0811 22:18:54.810835 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.810842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.810855 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:18:54.810905 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.810908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.810960 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.811049 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.815089 (+ 4040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.815956 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.817563 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.818143 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.818259 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.820240 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.820247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.820253 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.820292 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.822721 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.823386 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.823391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.823393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.823488 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.823825 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.823828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.823830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.824060 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.824185 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.825322 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.825462 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:18:54.827416 503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:54.828310 503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:18:54.831115 515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:18:54.831210 515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:18:54.835780 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.811427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.17.198.62:37175)
0811 22:18:54.811552 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.811560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.820784 (+ 9224us) server_negotiation.cc:407] Connection header received
0811 22:18:54.821519 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.821523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.821610 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.821688 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.823109 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.823990 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.827236 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.827662 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.827791 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.830153 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.830161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.830167 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.830200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.832334 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.833457 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.833462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.833464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.833572 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.833915 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.833919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.833921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.834247 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.834399 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.835280 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.835463 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:18:54.835054 388 master_runner.cc:186] Error getting master registration for 127.17.198.61:43435: OK, instance_id { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" instance_seqno: 1786486734470215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:18:54.839150 388 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:18:54.878744 381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } attempt: 4
Aug 11 22:18:54 dist-test-slave-77v9 krb5kdc[32661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486734, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:18:54.890419 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.879377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:35855 (local address 127.0.0.1:42326)
0811 22:18:54.879709 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.879727 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.879830 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.880498 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.880502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.880748 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:54.880914 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.880922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.882089 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.882093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.882961 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.882972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.883643 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.883647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.883740 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.884402 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:54.884418 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:54.886267 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:54.888538 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.888543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.888545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.888892 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.889212 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:54.889215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:54.889218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:54.889325 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:54.889795 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:54.889798 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:54.889931 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 22:18:54.890041 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:18:54.890211 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260811 22:18:54.890453 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.880031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.17.198.60:35855)
0811 22:18:54.880158 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.880163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.880177 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:18:54.880223 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.880226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.880275 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.880359 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.881075 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.881951 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.883104 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.883492 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.883611 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.886444 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.886448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.886451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:54.886493 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.888395 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.889000 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.889004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.889006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.889104 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:54.889437 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:54.889440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:54.889442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:54.889687 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.889789 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.890142 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.890277 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:18:54.896179 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: Bootstrap starting.
I20260811 22:18:54.899339 381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:54.900372 381 log.cc:824] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:54.903297 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: No bootstrap required, opened a new log
I20260811 22:18:54.909001 381 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.909360 381 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:54.909474 381 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaff209ba6864a35aa5a08e56429d545, State: Initialized, Role: FOLLOWER
I20260811 22:18:54.910094 381 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:54.910746 518 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:54.910910 518 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:54.911931 381 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:18:54.919157 530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:18:54.919382 530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:18:54.920393 32764 master_runner.cc:263] No leader master found from master eaff209ba6864a35aa5a08e56429d545
I20260811 22:18:54.920485 32764 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:18:54.960041 32701 tablet.cc:2369] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:18:54.966389 321 tablet.cc:2369] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:18:54.990752 413 tablet.cc:2369] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:55.151669 503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:55.151959 503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:55.153196 503 leader_election.cc:288] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72df51056a454dd9a0c15c986242661d (127.17.198.62:37175), eaff209ba6864a35aa5a08e56429d545 (127.17.198.61:43435)
I20260811 22:18:55.153725 32727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375e329f3f58472287b8b3407b8ef015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72df51056a454dd9a0c15c986242661d" is_pre_election: true
I20260811 22:18:55.153882 349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375e329f3f58472287b8b3407b8ef015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaff209ba6864a35aa5a08e56429d545" is_pre_election: true
I20260811 22:18:55.154052 32727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 375e329f3f58472287b8b3407b8ef015 in term 0.
I20260811 22:18:55.154247 349 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 375e329f3f58472287b8b3407b8ef015 in term 0.
I20260811 22:18:55.154561 411 leader_election.cc:302] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 375e329f3f58472287b8b3407b8ef015, 72df51056a454dd9a0c15c986242661d; no voters:
I20260811 22:18:55.154909 503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:55.155007 503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:55.155056 503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:55.156241 503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:55.156633 503 leader_election.cc:288] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [CANDIDATE]: Term 1 election: Requested vote from peers 72df51056a454dd9a0c15c986242661d (127.17.198.62:37175), eaff209ba6864a35aa5a08e56429d545 (127.17.198.61:43435)
I20260811 22:18:55.157076 349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375e329f3f58472287b8b3407b8ef015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaff209ba6864a35aa5a08e56429d545"
I20260811 22:18:55.157078 32727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375e329f3f58472287b8b3407b8ef015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72df51056a454dd9a0c15c986242661d"
I20260811 22:18:55.157239 349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:55.157256 32727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:55.158788 32727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 375e329f3f58472287b8b3407b8ef015 in term 1.
I20260811 22:18:55.158788 349 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 375e329f3f58472287b8b3407b8ef015 in term 1.
I20260811 22:18:55.159133 411 leader_election.cc:302] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 375e329f3f58472287b8b3407b8ef015, 72df51056a454dd9a0c15c986242661d; no voters:
I20260811 22:18:55.159330 503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:55.159557 503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [term 1 LEADER]: Becoming Leader. State: Replica: 375e329f3f58472287b8b3407b8ef015, State: Running, Role: LEADER
I20260811 22:18:55.159929 503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } }
I20260811 22:18:55.162288 533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e329f3f58472287b8b3407b8ef015. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.162478 533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:55.162959 535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:55.163846 535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:55.168797 349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [term 1 FOLLOWER]: Refusing update from remote peer 375e329f3f58472287b8b3407b8ef015: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:55.168886 32727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [term 1 FOLLOWER]: Refusing update from remote peer 375e329f3f58472287b8b3407b8ef015: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:55.169566 503 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:55.169721 533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:55.173893 476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e329f3f58472287b8b3407b8ef015. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.174042 476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:55.174047 518 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e329f3f58472287b8b3407b8ef015. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.174185 518 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:55.176712 541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.176760 503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.177042 503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:55.177201 541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:55.177736 476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.177897 476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:55.178903 518 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "375e329f3f58472287b8b3407b8ef015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72df51056a454dd9a0c15c986242661d" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37175 } } peers { permanent_uuid: "eaff209ba6864a35aa5a08e56429d545" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 43435 } } peers { permanent_uuid: "375e329f3f58472287b8b3407b8ef015" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 35855 } } }
I20260811 22:18:55.179037 518 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:18:55.179263 538 mvcc.cc:205] Tried to move back new op lower bound from 7317449667244142592 to 7317449667217874944. Current Snapshot: MvccSnapshot[applied={T|T < 7317449667244142592}]
I20260811 22:18:55.178310 535 catalog_manager.cc:1347] Generated new cluster ID: 30b9e15c80df4e7e8d81e3efd91696b5
I20260811 22:18:55.180059 535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:55.192122 535 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:55.193527 535 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:55.206306 535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375e329f3f58472287b8b3407b8ef015: Generated new TSK 0
I20260811 22:18:55.207301 535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:55.799520 491 catalog_manager.cc:1259] Loaded cluster ID: 30b9e15c80df4e7e8d81e3efd91696b5
I20260811 22:18:55.799646 491 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: loading cluster ID for follower catalog manager: success
I20260811 22:18:55.802556 491 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: acquiring CA information for follower catalog manager: success
I20260811 22:18:55.804402 491 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72df51056a454dd9a0c15c986242661d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:18:55.920996 530 catalog_manager.cc:1259] Loaded cluster ID: 30b9e15c80df4e7e8d81e3efd91696b5
I20260811 22:18:55.921108 530 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: loading cluster ID for follower catalog manager: success
I20260811 22:18:55.923063 530 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: acquiring CA information for follower catalog manager: success
I20260811 22:18:55.924595 530 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eaff209ba6864a35aa5a08e56429d545: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:18:56.290025 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32676
I20260811 22:18:56.298674 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 32764
I20260811 22:18:56.306782 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 388
2026-08-11T22:18:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3645 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:56 dist-test-slave-77v9 krb5kdc[550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:56 dist-test-slave-77v9 krb5kdc[550](info): set up 2 sockets
Aug 11 22:18:56 dist-test-slave-77v9 krb5kdc[550](info): commencing operation
krb5kdc: starting...
W20260811 22:18:57.475421 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:18:57.624162 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42781
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:57.746280 564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:57.746649 564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:57.746737 564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:57.751147 564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:57.751251 564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:57.751291 564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:57.751338 564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:57.751361 564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:57.758490 564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:57.759690 564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:57.760727 564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:57.770589 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:57.770589 570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:57.770638 569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:57.771064 564 server_base.cc:1034] running on GCE node
I20260811 22:18:57.771761 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:57.773085 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:57.774276 564 hybrid_clock.cc:648] HybridClock initialized: now 1786486737774254 us; error 39 us; skew 500 ppm
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486737, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:57.781626 564 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:18:57.782836 564 webserver.cc:460] Webserver started at http://127.17.198.62:36965/ using document root <none> and password file <none>
I20260811 22:18:57.783475 564 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:57.783550 564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:57.783742 564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:57.786558 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "15732d10693444b588dbd5401efb21bb"
format_stamp: "Formatted at 2026-08-11 22:18:57 on dist-test-slave-77v9"
server_key: "1c5a3217dbb7079a14e91a87d40767a9"
server_key_iv: "68649b1539ce085874db21e76222409a"
server_key_version: "encryptionkey@0"
I20260811 22:18:57.787175 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "15732d10693444b588dbd5401efb21bb"
format_stamp: "Formatted at 2026-08-11 22:18:57 on dist-test-slave-77v9"
server_key: "1c5a3217dbb7079a14e91a87d40767a9"
server_key_iv: "68649b1539ce085874db21e76222409a"
server_key_version: "encryptionkey@0"
I20260811 22:18:57.791628 564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:18:57.794700 581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:57.796226 564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:57.796428 564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "15732d10693444b588dbd5401efb21bb"
format_stamp: "Formatted at 2026-08-11 22:18:57 on dist-test-slave-77v9"
server_key: "1c5a3217dbb7079a14e91a87d40767a9"
server_key_iv: "68649b1539ce085874db21e76222409a"
server_key_version: "encryptionkey@0"
I20260811 22:18:57.796577 564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:57.825435 564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:57.829078 564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:57.829319 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:57.837734 564 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:42781
I20260811 22:18:57.837783 640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:42781 every 8 connection(s)
I20260811 22:18:57.838869 564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:57.842010 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:57.843179 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 564
I20260811 22:18:57.843317 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:18:57.843609 18200 external_mini_cluster.cc:1428] Setting key 3670183df19d2db03ec330adfe2d4d83
I20260811 22:18:57.849253 641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb: Bootstrap starting.
I20260811 22:18:57.851771 641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:57.852586 641 log.cc:824] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:57.855386 641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb: No bootstrap required, opened a new log
I20260811 22:18:57.859995 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:57.845128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.17.198.62:42781)
0811 22:18:57.845595 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:57.845613 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:57.845656 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:18:57.846326 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:57.846343 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:57.846705 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:57.847011 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:57.848070 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:57.849288 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:57.850311 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:57.850821 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:57.851017 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:57.853980 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:57.853988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:57.854009 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:57.854043 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:57.856183 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:57.856710 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:57.856717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:57.856724 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:57.856907 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:57.857273 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:57.857276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:57.857278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:57.857906 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:57.858184 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:57.858523 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:18:57.858886 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":156,"threads_started":1}
I20260811 22:18:57.860484 641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } }
I20260811 22:18:57.860733 641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:57.860797 641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15732d10693444b588dbd5401efb21bb, State: Initialized, Role: FOLLOWER
I20260811 22:18:57.861311 641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } }
I20260811 22:18:57.861518 641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:57.861588 641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:57.861692 641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:57.863031 641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } }
I20260811 22:18:57.863440 641 leader_election.cc:302] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15732d10693444b588dbd5401efb21bb; no voters:
I20260811 22:18:57.863880 641 leader_election.cc:288] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:57.864073 646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:57.864207 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:57.864357 646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [term 1 LEADER]: Becoming Leader. State: Replica: 15732d10693444b588dbd5401efb21bb, State: Running, Role: LEADER
I20260811 22:18:57.864722 646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } }
I20260811 22:18:57.865437 641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:57.866140 647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15732d10693444b588dbd5401efb21bb. Latest consensus state: current_term: 1 leader_uuid: "15732d10693444b588dbd5401efb21bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } } }
I20260811 22:18:57.866290 647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:57.866757 656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:57.867075 648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15732d10693444b588dbd5401efb21bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732d10693444b588dbd5401efb21bb" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42781 } } }
I20260811 22:18:57.867173 648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:57.870039 656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:57.876711 656 catalog_manager.cc:1347] Generated new cluster ID: 488614693ffd4ea18d0997cc9a37471c
I20260811 22:18:57.876856 656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:57.892509 656 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:57.893734 656 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:57.915122 656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15732d10693444b588dbd5401efb21bb: Generated new TSK 0
I20260811 22:18:57.916038 656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:57.921768 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 564
2026-08-11T22:18:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1613 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[672](info): set up 2 sockets
Aug 11 22:18:57 dist-test-slave-77v9 krb5kdc[672](info): commencing operation
krb5kdc: starting...
W20260811 22:18:59.097091 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:18:59.249099 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:59.367780 688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:59.368139 688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:59.368216 688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:59.372430 688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:59.372548 688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:59.372584 688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:59.372627 688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:59.372659 688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:59.378309 688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:59.379477 688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:59.380553 688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:59.389529 693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.389586 697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.389586 694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:59.389742 688 server_base.cc:1034] running on GCE node
I20260811 22:18:59.390614 688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:59.391726 688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:59.392899 688 hybrid_clock.cc:648] HybridClock initialized: now 1786486739392876 us; error 35 us; skew 500 ppm
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:59.395913 688 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:18:59.396997 688 webserver.cc:460] Webserver started at http://127.17.198.62:37843/ using document root <none> and password file <none>
I20260811 22:18:59.397547 688 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:59.397608 688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:59.397779 688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:59.399904 688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "8a9e110b184542c28c5dad6936b2ba94"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "e3b251f3d21715548248fc3078de190f"
server_key_iv: "14420d1bd08892ca381ebdf0303376d1"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.400465 688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "8a9e110b184542c28c5dad6936b2ba94"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "e3b251f3d21715548248fc3078de190f"
server_key_iv: "14420d1bd08892ca381ebdf0303376d1"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.404531 688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:18:59.407438 705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.409133 688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:18:59.409313 688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "8a9e110b184542c28c5dad6936b2ba94"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "e3b251f3d21715548248fc3078de190f"
server_key_iv: "14420d1bd08892ca381ebdf0303376d1"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.409457 688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:59.422673 688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:59.426872 688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:59.427160 688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:59.437477 688 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35391
I20260811 22:18:59.438192 766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35391 every 8 connection(s)
I20260811 22:18:59.439035 688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:59.442303 767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:59.448872 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 688
I20260811 22:18:59.449020 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:18:59.449129 767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap starting.
I20260811 22:18:59.449376 18200 external_mini_cluster.cc:1428] Setting key c9987bd9f83d3f7ea862d61a52f43325
I20260811 22:18:59.452116 767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:59.453063 767 log.cc:824] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:59.455191 767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: No bootstrap required, opened a new log
I20260811 22:18:59.459090 767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:18:59.459443 767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:59.459502 767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Initialized, Role: FOLLOWER
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:59.460059 767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:18:59.460268 767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:59.460348 767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:59.460457 767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:59.461774 767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:18:59.462199 767 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e110b184542c28c5dad6936b2ba94; no voters:
I20260811 22:18:59.462611 767 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:59.462702 772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:59.462920 772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 LEADER]: Becoming Leader. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Running, Role: LEADER
I20260811 22:18:59.463250 772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:18:59.464053 767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:59.464529 772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e110b184542c28c5dad6936b2ba94. Latest consensus state: current_term: 1 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:18:59.464663 772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:59.465075 774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:18:59.465200 774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:59.467867 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:59.451111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56894 (local address 127.17.198.62:35391)
0811 22:18:59.451658 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:59.451674 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:59.451713 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:18:59.452631 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:59.452653 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:59.453174 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:59.453596 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:59.454557 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.455738 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.456717 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.457345 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.457541 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:59.460879 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:59.460888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:59.460909 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:59.460945 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:59.463215 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.463804 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.463815 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.463825 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.463998 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.464372 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.464377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.464379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.465316 (+ 937us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:59.465536 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:59.466002 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 22:18:59.466359 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":179,"threads_started":1}
I20260811 22:18:59.466691 777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:59.471550 777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:59.478085 777 catalog_manager.cc:1347] Generated new cluster ID: 143a0639f73745fd991b3fc2f2c330dc
I20260811 22:18:59.478226 777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:59.491102 777 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:59.492178 777 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:59.502058 777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Generated new TSK 0
I20260811 22:18:59.502920 777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:18:59.549397 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:59.671308 800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:59.671655 800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:59.671703 800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:59.676179 800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:59.676308 800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:59.676448 800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:18:59.682675 800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:59.683902 800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:59.685027 800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:59.695896 806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:59.696013 800 server_base.cc:1034] running on GCE node
W20260811 22:18:59.696172 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.695896 805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:59.696717 800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:59.697547 800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:59.698762 800 hybrid_clock.cc:648] HybridClock initialized: now 1786486739698715 us; error 69 us; skew 500 ppm
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:59.702103 800 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:18:59.703217 800 webserver.cc:460] Webserver started at http://127.17.198.1:45295/ using document root <none> and password file <none>
I20260811 22:18:59.703889 800 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:59.704007 800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:59.704314 800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:59.706851 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "67dbc860a99d4115938224c8b9f0846f"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "0daa1b3cefc24dd43b50081edebb7731"
server_key_iv: "1c6592a6216e4867f714c0ebf484da16"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.707557 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "67dbc860a99d4115938224c8b9f0846f"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "0daa1b3cefc24dd43b50081edebb7731"
server_key_iv: "1c6592a6216e4867f714c0ebf484da16"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.712209 800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:18:59.715510 816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.717303 800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:59.717501 800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "67dbc860a99d4115938224c8b9f0846f"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "0daa1b3cefc24dd43b50081edebb7731"
server_key_iv: "1c6592a6216e4867f714c0ebf484da16"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.717659 800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:59.741876 800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:59.746069 800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:59.746351 800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:59.747145 800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:59.748377 800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:59.748458 800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.748543 800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:59.748600 800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.759258 800 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:41823
I20260811 22:18:59.759296 944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:41823 every 8 connection(s)
I20260811 22:18:59.760321 800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:59.768682 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 800
I20260811 22:18:59.768870 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:59.769241 18200 external_mini_cluster.cc:1428] Setting key 27803116c5e867fe117a2234f4915d1b
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:18:59.779134 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:59.762670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38161 (local address 127.17.198.62:35391)
0811 22:18:59.762894 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:59.762899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:59.764242 (+ 1343us) server_negotiation.cc:407] Connection header received
0811 22:18:59.766315 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:59.766319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:59.766380 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:59.766552 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:59.768058 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.768964 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.770447 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.770789 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.770910 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:59.774866 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:59.774870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:59.774875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:59.774918 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:59.776599 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.777372 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.777376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.777378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.777484 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.777896 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.777899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.777901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.778116 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:59.778230 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:59.778806 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:18:59.778959 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:18:59.780160 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:59.763226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:38161)
0811 22:18:59.764040 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:59.764109 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:59.766040 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:59.766785 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:59.766793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:59.767300 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:59.767871 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:59.767887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.769126 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.769129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:59.770309 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:59.770321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.771010 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.771015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:59.771260 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:59.772052 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:59.772082 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:59.774604 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:59.776799 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:59.776858 (+ 59us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:59.776882 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:59.777232 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:59.777593 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:59.777596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:59.777598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:59.777778 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:59.778232 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:59.778242 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:59.778540 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 22:18:59.778747 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:18:59.779149 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":144,"threads_started":1}
I20260811 22:18:59.781905 945 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:18:59.782259 945 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:59.782979 945 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
I20260811 22:18:59.785275 719 ts_manager.cc:194] Registered new tserver with Master: 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:18:59.786818 719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:38161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:18:59.809293 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:59.934140 951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:59.934504 951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:59.934609 951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:59.938973 951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:59.939118 951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:59.939224 951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:18:59.945120 951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:18:59.946529 951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:59.947667 951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:59.958570 957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.958573 956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.958573 959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:59.958976 951 server_base.cc:1034] running on GCE node
I20260811 22:18:59.959430 951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:59.960140 951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:59.961345 951 hybrid_clock.cc:648] HybridClock initialized: now 1786486739961327 us; error 33 us; skew 500 ppm
Aug 11 22:18:59 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:59.964286 951 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:18:59.965494 951 webserver.cc:460] Webserver started at http://127.17.198.2:46461/ using document root <none> and password file <none>
I20260811 22:18:59.966145 951 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:59.966269 951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:59.966542 951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:59.968777 951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "e889c3b6dcad43088aee18d0d92399b8"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "62cb9730c758c8c288d002a2eb66845e"
server_key_iv: "abd3368a55ec62dab862075fa924bd75"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.969460 951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "e889c3b6dcad43088aee18d0d92399b8"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "62cb9730c758c8c288d002a2eb66845e"
server_key_iv: "abd3368a55ec62dab862075fa924bd75"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.974229 951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:18:59.977003 967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.978432 951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:59.978569 951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "e889c3b6dcad43088aee18d0d92399b8"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "62cb9730c758c8c288d002a2eb66845e"
server_key_iv: "abd3368a55ec62dab862075fa924bd75"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.978672 951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.004737 951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.008760 951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.009038 951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.009734 951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:00.010669 951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:00.010725 951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.010771 951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:00.010807 951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.021462 951 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43039
I20260811 22:19:00.021546 1102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43039 every 8 connection(s)
I20260811 22:19:00.022657 951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:00.028610 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 951
I20260811 22:19:00.028767 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:00.029177 18200 external_mini_cluster.cc:1428] Setting key 48e1bd1aed72e2e8a2fa2888c14cae74
Aug 11 22:19:00 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:00.039264 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.024669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:38519 (local address 127.17.198.62:35391)
0811 22:19:00.024875 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.024885 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.025935 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 22:19:00.027421 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.027425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.027482 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.027569 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:00.029148 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.029750 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.031338 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.031567 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.031666 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.034867 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:00.034871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:00.034873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:00.034901 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:00.036674 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.037475 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.037481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.037487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.037575 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.037978 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.037981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.037983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.038207 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:00.038326 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.038915 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.039065 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:19:00.040225 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.025208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:38519)
0811 22:19:00.025729 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.025777 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.027152 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.027758 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.027765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.028161 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:00.028955 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.028978 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.029874 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.029878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.031149 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.031165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.031758 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.031765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.032031 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.032786 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:00.032844 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:00.034629 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:00.036991 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.036997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.037012 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.037344 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.037679 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.037682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.037684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.037856 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.038328 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:00.038339 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:00.038639 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.038858 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.039242 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
I20260811 22:19:00.041896 1104 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
I20260811 22:19:00.042200 1104 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:00.042804 1104 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:19:00.044272 719 ts_manager.cc:194] Registered new tserver with Master: e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039)
I20260811 22:19:00.045081 719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:38519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:19:00.066437 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:00.185532 1110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:00.185878 1110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:00.185962 1110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:00.190457 1110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:00.190564 1110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:00.190662 1110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:19:00.196441 1110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:00.197739 1110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:00.198882 1110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:00.209810 1117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.209817 1116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.209808 1119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:00.210110 1110 server_base.cc:1034] running on GCE node
I20260811 22:19:00.210709 1110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:00.211445 1110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:00.212630 1110 hybrid_clock.cc:648] HybridClock initialized: now 1786486740212601 us; error 44 us; skew 500 ppm
Aug 11 22:19:00 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:00.216106 1110 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:19:00.217240 1110 webserver.cc:460] Webserver started at http://127.17.198.3:46655/ using document root <none> and password file <none>
I20260811 22:19:00.217873 1110 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:00.217972 1110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:00.218222 1110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:00.220471 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "e679a73a13244e5c881d28b77b69859a"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-77v9"
server_key: "a09b1c2a2bc2f20a2bfc9257e4260d1c"
server_key_iv: "0d07e6e5119fafd07962262877621b97"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.221153 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "e679a73a13244e5c881d28b77b69859a"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-77v9"
server_key: "a09b1c2a2bc2f20a2bfc9257e4260d1c"
server_key_iv: "0d07e6e5119fafd07962262877621b97"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.225786 1110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:00.228605 1126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.230150 1110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:00.230314 1110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "e679a73a13244e5c881d28b77b69859a"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-77v9"
server_key: "a09b1c2a2bc2f20a2bfc9257e4260d1c"
server_key_iv: "0d07e6e5119fafd07962262877621b97"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.230463 1110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.266566 1110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.270275 1110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.270514 1110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.271165 1110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:00.272059 1110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:00.272133 1110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.272209 1110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:00.272253 1110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.281774 1110 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:35803
I20260811 22:19:00.281792 1258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:35803 every 8 connection(s)
I20260811 22:19:00.282869 1110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:00.285432 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1110
I20260811 22:19:00.285564 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:00.285923 18200 external_mini_cluster.cc:1428] Setting key 8ab1360001e8d82001d6b87dce0c2736
Aug 11 22:19:00 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:00.298683 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.284947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:47113 (local address 127.17.198.62:35391)
0811 22:19:00.285110 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.285114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.286252 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 22:19:00.287942 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.287946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.288004 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.288107 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:00.289361 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.289896 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.291135 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.291366 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.291474 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.294294 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:00.294297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:00.294299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:00.294326 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:00.296186 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.296899 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.296903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.296904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.296997 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.297417 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.297420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.297422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.297638 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:00.297758 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.298340 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.298491 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:19:00.299693 1262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.285435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:47113)
0811 22:19:00.286092 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.286138 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.287725 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.288284 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.288290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.288659 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:00.289217 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.289230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.290061 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.290069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.290967 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.290977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.291529 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.291533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.291714 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.292349 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:00.292376 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:00.294099 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:00.296373 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.296380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.296400 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.296719 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.297106 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.297110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.297113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.297293 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.297758 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:00.297768 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:00.298062 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.298271 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.298667 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":156,"threads_started":1}
I20260811 22:19:00.300894 1259 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
I20260811 22:19:00.301169 1259 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:00.301712 1259 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
I20260811 22:19:00.302798 719 ts_manager.cc:194] Registered new tserver with Master: e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
I20260811 22:19:00.303349 719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:47113
I20260811 22:19:00.312594 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:19:00.323976 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.315162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.17.198.62:35391)
0811 22:19:00.315331 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.315336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.315565 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:19:00.315654 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.315656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.315704 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.315796 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:00.316627 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.317336 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.318198 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.318434 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.318528 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.320060 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:00.320064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:00.320066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:00.320091 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:00.321758 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.322413 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.322415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.322417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.322503 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.322892 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.322895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.322897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.323194 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:00.323307 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.323553 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.323663 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:19:00.327175 719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:00.330371 719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:00.346872 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.340233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.0.0.1:58844)
0811 22:19:00.340954 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.341016 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.341166 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.342191 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.342197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.342220 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:00.342377 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.342401 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.343980 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.343984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.345205 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.345216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346378 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.346467 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346506 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.346562 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.346730 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":24,"thread_start_us":113,"threads_started":1}
I20260811 22:19:00.346881 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.341535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.17.198.3:35803)
0811 22:19:00.342072 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.342090 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.342114 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:19:00.342192 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.342199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.342387 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.342654 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:00.342828 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.343583 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.345452 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346096 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346226 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346388 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.346599 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.346704 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":183,"threads_started":1}
I20260811 22:19:00.347043 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.340655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.0.0.1:36174)
0811 22:19:00.341764 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.341779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.341874 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.342504 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.342509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.342533 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:00.342817 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.342823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.344176 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.344179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.345352 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.345362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346621 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.346735 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346756 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.346824 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.346900 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":5,"thread_start_us":108,"threads_started":1}
I20260811 22:19:00.347157 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.341011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.17.198.1:41823)
0811 22:19:00.341623 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.341632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.341676 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:19:00.341762 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.341770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.342008 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.342541 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:00.342688 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.343859 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.345353 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346221 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346689 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346837 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.346897 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.346992 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":107,"threads_started":1}
I20260811 22:19:00.347631 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.341098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.17.198.2:43039)
0811 22:19:00.342052 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.342069 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.342097 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:19:00.342177 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.342182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.342347 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.342514 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:00.342940 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.344038 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.345499 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346287 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346397 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346561 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.347425 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.347511 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":128,"threads_started":1}
I20260811 22:19:00.348371 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.340493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:35803 (local address 127.0.0.1:51344)
0811 22:19:00.341258 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.341273 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.341366 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.342556 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.342559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.342581 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:00.342687 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.342696 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.343722 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.343727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.345325 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.345336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.346229 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.346233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.346357 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.346396 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.346552 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.348228 (+ 1676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":44,"spinlock_wait_cycles":768,"thread_start_us":115,"threads_started":1}
I20260811 22:19:00.350409 1024 tablet_service.cc:1467] Processing CreateTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 (DEFAULT_TABLE table=test-table [id=74fe4b25e2b744f9807df60f8a23f358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:00.350455 871 tablet_service.cc:1467] Processing CreateTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 (DEFAULT_TABLE table=test-table [id=74fe4b25e2b744f9807df60f8a23f358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:00.351456 1024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fea480280f4ac9b3e05d83696af4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:00.351533 871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fea480280f4ac9b3e05d83696af4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:00.351361 1181 tablet_service.cc:1467] Processing CreateTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 (DEFAULT_TABLE table=test-table [id=74fe4b25e2b744f9807df60f8a23f358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:00.352370 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fea480280f4ac9b3e05d83696af4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:00.358250 1278 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Bootstrap starting.
I20260811 22:19:00.359812 1279 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Bootstrap starting.
I20260811 22:19:00.359905 1280 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Bootstrap starting.
I20260811 22:19:00.361408 1278 tablet_bootstrap.cc:654] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:00.361833 1280 tablet_bootstrap.cc:654] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:00.362190 1279 tablet_bootstrap.cc:654] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:00.362346 1278 log.cc:824] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:00.363029 1279 log.cc:824] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:00.363047 1280 log.cc:824] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:00.365284 1279 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: No bootstrap required, opened a new log
I20260811 22:19:00.365324 1278 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: No bootstrap required, opened a new log
I20260811 22:19:00.365338 1280 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: No bootstrap required, opened a new log
I20260811 22:19:00.365525 1279 ts_tablet_manager.cc:1397] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260811 22:19:00.365527 1278 ts_tablet_manager.cc:1397] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 22:19:00.365527 1280 ts_tablet_manager.cc:1397] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:19:00.369246 1279 raft_consensus.cc:348] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.369551 1279 raft_consensus.cc:374] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:00.369618 1279 raft_consensus.cc:729] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dbc860a99d4115938224c8b9f0846f, State: Initialized, Role: FOLLOWER
I20260811 22:19:00.369582 1280 raft_consensus.cc:348] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.369853 1280 raft_consensus.cc:374] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:00.369582 1278 raft_consensus.cc:348] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.369913 1280 raft_consensus.cc:729] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e679a73a13244e5c881d28b77b69859a, State: Initialized, Role: FOLLOWER
I20260811 22:19:00.369961 1278 raft_consensus.cc:374] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:00.370019 1278 raft_consensus.cc:729] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e889c3b6dcad43088aee18d0d92399b8, State: Initialized, Role: FOLLOWER
I20260811 22:19:00.370118 1279 consensus_queue.cc:260] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.370402 1278 consensus_queue.cc:260] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.370399 1280 consensus_queue.cc:260] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.371289 1279 ts_tablet_manager.cc:1428] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:19:00.371359 1278 ts_tablet_manager.cc:1428] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:19:00.371362 1280 ts_tablet_manager.cc:1428] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:19:00.371713 1104 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
I20260811 22:19:00.371716 945 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
I20260811 22:19:00.371789 1259 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
I20260811 22:19:00.421643 1287 raft_consensus.cc:482] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:00.421867 1287 raft_consensus.cc:504] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.423233 1287 leader_election.cc:288] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823), e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039)
I20260811 22:19:00.428772 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.423702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:54813 (local address 127.17.198.1:41823)
0811 22:19:00.423878 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.423883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.423898 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:19:00.423963 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.423965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.424003 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.424104 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:00.424411 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.425241 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.427065 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.427814 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.427986 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.428047 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.428524 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.428623 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 22:19:00.428772 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.423726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:52351 (local address 127.17.198.2:43039)
0811 22:19:00.423871 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.423877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.424075 (+ 198us) server_negotiation.cc:407] Connection header received
0811 22:19:00.424223 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.424226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.424313 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.424416 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:00.424687 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.425419 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.427066 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.427880 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.428003 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.428048 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.428524 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.428623 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:19:00.428802 1262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.423467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.17.198.3:54813)
0811 22:19:00.423588 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.423607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.423711 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.424146 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.424151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.424170 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:00.424291 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.424296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.425360 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.425362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.426893 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.426905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.428109 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.428113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.428306 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.428343 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.428441 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.428569 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":76}
I20260811 22:19:00.428833 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.423526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.17.198.3:52351)
0811 22:19:00.423964 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.423976 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.424103 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.424465 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.424467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.424483 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:00.424569 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.424574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.425620 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.425623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.426894 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.426905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.428109 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.428113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.428301 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.428354 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.428441 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.428569 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":124,"thread_start_us":156,"threads_started":1}
I20260811 22:19:00.429713 895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fea480280f4ac9b3e05d83696af4b9" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dbc860a99d4115938224c8b9f0846f" is_pre_election: true
I20260811 22:19:00.429713 1045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fea480280f4ac9b3e05d83696af4b9" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e889c3b6dcad43088aee18d0d92399b8" is_pre_election: true
I20260811 22:19:00.430105 895 raft_consensus.cc:2454] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e679a73a13244e5c881d28b77b69859a in term 0.
I20260811 22:19:00.430105 1045 raft_consensus.cc:2454] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e679a73a13244e5c881d28b77b69859a in term 0.
I20260811 22:19:00.430698 1127 leader_election.cc:302] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f, e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8; no voters:
I20260811 22:19:00.431133 1287 raft_consensus.cc:2781] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:00.431267 1287 raft_consensus.cc:482] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:00.431329 1287 raft_consensus.cc:3037] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:00.432649 1287 raft_consensus.cc:504] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.433099 1287 leader_election.cc:288] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 election: Requested vote from peers 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823), e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039)
I20260811 22:19:00.433637 1045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fea480280f4ac9b3e05d83696af4b9" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e889c3b6dcad43088aee18d0d92399b8"
I20260811 22:19:00.433734 895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fea480280f4ac9b3e05d83696af4b9" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dbc860a99d4115938224c8b9f0846f"
I20260811 22:19:00.433811 1045 raft_consensus.cc:3037] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:00.433871 895 raft_consensus.cc:3037] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:00.435324 895 raft_consensus.cc:2454] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e679a73a13244e5c881d28b77b69859a in term 1.
I20260811 22:19:00.435319 1045 raft_consensus.cc:2454] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e679a73a13244e5c881d28b77b69859a in term 1.
I20260811 22:19:00.435660 1127 leader_election.cc:302] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8; no voters:
I20260811 22:19:00.436021 1287 raft_consensus.cc:2781] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:00.436288 1287 raft_consensus.cc:686] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 1 LEADER]: Becoming Leader. State: Replica: e679a73a13244e5c881d28b77b69859a, State: Running, Role: LEADER
I20260811 22:19:00.436655 1287 consensus_queue.cc:237] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:00.440685 717 catalog_manager.cc:5495] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a reported cstate change: term changed from 0 to 1, leader changed from <none> to e679a73a13244e5c881d28b77b69859a (127.17.198.3). New cstate: current_term: 1 leader_uuid: "e679a73a13244e5c881d28b77b69859a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:00.476225 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.471961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.17.198.3:35803)
0811 22:19:00.472146 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.472151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.472168 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:19:00.472342 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.472345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.472414 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.472549 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:00.472858 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.473543 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.474513 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.474765 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.474891 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.475263 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:00.475739 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:00.475891 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.475965 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.476029 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:19:00.482884 895 raft_consensus.cc:1261] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Refusing update from remote peer e679a73a13244e5c881d28b77b69859a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:00.482930 1045 raft_consensus.cc:1261] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Refusing update from remote peer e679a73a13244e5c881d28b77b69859a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:00.483865 1287 consensus_queue.cc:1035] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [LEADER]: Connected to new peer: Peer: permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:00.484114 1290 consensus_queue.cc:1035] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:00.495469 1296 mvcc.cc:205] Tried to move back new op lower bound from 7317449689008644096 to 7317449688831463424. Current Snapshot: MvccSnapshot[applied={T|T < 7317449689008644096}]
I20260811 22:19:00.504881 717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:00.505131 717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:00.508435 717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Sending DeleteTablet for 3 replicas of tablet 20fea480280f4ac9b3e05d83696af4b9
I20260811 22:19:00.509555 1181 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:00 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:51344
I20260811 22:19:00.509677 871 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:00 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:58844
I20260811 22:19:00.509773 1024 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:00 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36174
I20260811 22:19:00.510390 1308 tablet_replica.cc:330] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: stopping tablet replica
I20260811 22:19:00.510493 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 800
I20260811 22:19:00.510838 1308 raft_consensus.cc:2229] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:00.511256 1307 tablet_replica.cc:330] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: stopping tablet replica
I20260811 22:19:00.511565 1307 raft_consensus.cc:2229] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:00.512177 1307 raft_consensus.cc:2258] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:00.512948 1309 tablet_replica.cc:330] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: stopping tablet replica
I20260811 22:19:00.513273 1309 raft_consensus.cc:2229] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:00.513639 1309 raft_consensus.cc:2258] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:00.515028 1307 ts_tablet_manager.cc:1905] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:00.515265 1309 ts_tablet_manager.cc:1905] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:00.518239 1309 ts_tablet_manager.cc:1918] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:00.518239 1307 ts_tablet_manager.cc:1918] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:00.518385 1309 log.cc:1196] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/20fea480280f4ac9b3e05d83696af4b9
I20260811 22:19:00.518617 1307 log.cc:1196] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/20fea480280f4ac9b3e05d83696af4b9
I20260811 22:19:00.518759 1309 ts_tablet_manager.cc:1939] T 20fea480280f4ac9b3e05d83696af4b9 P e889c3b6dcad43088aee18d0d92399b8: Deleting consensus metadata
I20260811 22:19:00.519160 1307 ts_tablet_manager.cc:1939] T 20fea480280f4ac9b3e05d83696af4b9 P e679a73a13244e5c881d28b77b69859a: Deleting consensus metadata
I20260811 22:19:00.520453 707 catalog_manager.cc:4841] TS e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): tablet 20fea480280f4ac9b3e05d83696af4b9 (table test-table [id=74fe4b25e2b744f9807df60f8a23f358]) successfully deleted
I20260811 22:19:00.520453 706 catalog_manager.cc:4841] TS e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): tablet 20fea480280f4ac9b3e05d83696af4b9 (table test-table [id=74fe4b25e2b744f9807df60f8a23f358]) successfully deleted
W20260811 22:19:00.524950 709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:41823 (error 108)
I20260811 22:19:00.525699 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 951
I20260811 22:19:00.528749 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.525872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.0.0.1:58858)
0811 22:19:00.528512 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.528617 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2495}
W20260811 22:19:00.529162 709 catalog_manager.cc:4584] TS 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20fea480280f4ac9b3e05d83696af4b9: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41823: connect: Connection refused (error 111)
I20260811 22:19:00.535981 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 1110
I20260811 22:19:00.545734 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 688
I20260811 22:19:00.555073 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--webserver_interface=127.17.198.62
--webserver_port=37843
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:00.680104 1312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:00.680444 1312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:00.680493 1312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:00.685189 1312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:00.685286 1312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:00.685309 1312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:00.685328 1312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:00.685346 1312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:00.691571 1312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=37843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:00.692718 1312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:00.693769 1312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:00.714651 1321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.714630 1322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.714630 1325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:00.715101 1312 server_base.cc:1034] running on GCE node
I20260811 22:19:00.716194 1312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:00.717819 1312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:00.719045 1312 hybrid_clock.cc:648] HybridClock initialized: now 1786486740719002 us; error 66 us; skew 500 ppm
Aug 11 22:19:00 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:00.722818 1312 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:19:00.723935 1312 webserver.cc:460] Webserver started at http://127.17.198.62:37843/ using document root <none> and password file <none>
I20260811 22:19:00.724576 1312 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:00.724699 1312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:00.729174 1312 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:00.731667 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.733193 1312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:00.733358 1312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "8a9e110b184542c28c5dad6936b2ba94"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "e3b251f3d21715548248fc3078de190f"
server_key_iv: "14420d1bd08892ca381ebdf0303376d1"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.734019 1312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.753628 1312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.757364 1312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.757552 1312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.765830 1312 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35391
I20260811 22:19:00.766289 1395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35391 every 8 connection(s)
I20260811 22:19:00.767565 1312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:00.773159 1396 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:00.774364 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1312
I20260811 22:19:00.774967 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:41823
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=45295
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:00.775745 1396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap starting.
I20260811 22:19:00.796955 1396 log.cc:824] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:00.807122 1396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:00.807650 1396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap complete.
I20260811 22:19:00.812688 1396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:00.813279 1396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Initialized, Role: FOLLOWER
I20260811 22:19:00.813987 1396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:00.814165 1396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:00.814280 1396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:00.814433 1396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:19:00.816920 1396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:00.817381 1396 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e110b184542c28c5dad6936b2ba94; no voters:
I20260811 22:19:00.817812 1396 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:19:00.817930 1400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:19:00.818185 1400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 LEADER]: Becoming Leader. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Running, Role: LEADER
I20260811 22:19:00.818542 1400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:00.819314 1396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:00.819705 1400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e110b184542c28c5dad6936b2ba94. Latest consensus state: current_term: 2 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:19:00.819839 1400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:00.820156 1401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:19:00.820277 1401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:00.820385 1404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:00.826427 1404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:00.827346 1404 catalog_manager.cc:1259] Loaded cluster ID: 143a0639f73745fd991b3fc2f2c330dc
I20260811 22:19:00.827422 1404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:00.828781 1404 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:19:00.832063 1404 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:19:00.837647 1404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:00.838650 1404 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Loaded TSK: 0
I20260811 22:19:00.840004 1404 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:19:00.922549 1398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:00.922865 1398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:00.922948 1398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:00.927368 1398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:00.927471 1398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:00.927537 1398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:19:00.933584 1398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:41823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=45295
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:00.934785 1398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:00.935875 1398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:00.946648 1428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.946648 1426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.946666 1430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:00.947268 1398 server_base.cc:1034] running on GCE node
I20260811 22:19:00.947901 1398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:00.948902 1398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:00.950125 1398 hybrid_clock.cc:648] HybridClock initialized: now 1786486740950081 us; error 63 us; skew 500 ppm
Aug 11 22:19:00 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:00.954471 1398 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:19:00.955928 1398 webserver.cc:460] Webserver started at http://127.17.198.1:45295/ using document root <none> and password file <none>
I20260811 22:19:00.956748 1398 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:00.956916 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:00.963028 1398 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:19:00.966573 1439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.968509 1398 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 22:19:00.968688 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "67dbc860a99d4115938224c8b9f0846f"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "0daa1b3cefc24dd43b50081edebb7731"
server_key_iv: "1c6592a6216e4867f714c0ebf484da16"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.969525 1398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.993917 1398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.998016 1398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.998268 1398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.999042 1398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:01.000361 1447 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:19:01.004799 1398 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:19:01.004925 1398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:19:01.005002 1398 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:19:01.006867 1398 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:19:01.006942 1398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:19:01.007202 1447 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Bootstrap starting.
I20260811 22:19:01.019239 1398 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:41823
I20260811 22:19:01.019311 1574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:41823 every 8 connection(s)
I20260811 22:19:01.020932 1398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:01.025379 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1398
I20260811 22:19:01.025961 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:43039
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=46461
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:01.030531 1447 log.cc:824] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:01.042237 1447 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:01.042774 1447 tablet_bootstrap.cc:492] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Bootstrap complete.
I20260811 22:19:01.043566 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.023570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:42179 (local address 127.17.198.62:35391)
0811 22:19:01.023984 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.024007 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.025348 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 22:19:01.027305 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.027327 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.027909 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.028404 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.029946 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.031258 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.032949 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.033493 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.033718 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.036931 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.036938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.036952 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.036992 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.039492 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.040216 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.040223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.040228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.040395 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.040882 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.040885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.040886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.041308 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.041474 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.042100 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.042435 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":157,"threads_started":1}
I20260811 22:19:01.043589 1578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.023609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:42179)
0811 22:19:01.025057 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.025144 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.026641 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.028494 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.028505 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.029118 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:01.029673 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.029692 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.031489 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.031494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.032649 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.032663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.033688 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.033693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.033939 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.034670 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:01.034690 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:01.036544 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:01.039670 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.039677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.039693 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.040041 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.040522 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.040526 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.040529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.040706 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.041447 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:01.041455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:01.041732 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.041943 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.042512 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"thread_start_us":186,"threads_started":1}
I20260811 22:19:01.044456 1447 ts_tablet_manager.cc:1397] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Time spent bootstrapping tablet: real 0.037s user 0.021s sys 0.012s
I20260811 22:19:01.046985 1575 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
I20260811 22:19:01.047312 1575 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:01.047927 1575 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
I20260811 22:19:01.047793 1447 raft_consensus.cc:348] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:01.048409 1447 raft_consensus.cc:729] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dbc860a99d4115938224c8b9f0846f, State: Initialized, Role: FOLLOWER
I20260811 22:19:01.049000 1447 consensus_queue.cc:260] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } }
I20260811 22:19:01.050316 1447 ts_tablet_manager.cc:1428] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
I20260811 22:19:01.052587 1353 ts_manager.cc:194] Registered new tserver with Master: 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823)
I20260811 22:19:01.056594 1353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:42179
I20260811 22:19:01.059624 1575 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.1@KRBTEST.COM
I20260811 22:19:01.069380 1583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.056085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.0.0.1:58860)
0811 22:19:01.057094 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.057145 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.057373 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.057976 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.057981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.058400 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:01.058513 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.058520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.059526 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.059531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.060413 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.060422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.061122 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.061125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.061256 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.061950 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:01.061962 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:01.064147 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:01.066978 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.066993 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.067001 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.067351 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.067766 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.067770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.067772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.067908 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.068503 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:01.068511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:01.068666 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.068855 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.069147 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":156,"threads_started":1}
I20260811 22:19:01.069380 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.056222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.17.198.1:41823)
0811 22:19:01.056895 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.056918 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.057282 (+ 364us) server_negotiation.cc:407] Connection header received
0811 22:19:01.057554 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.057562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.057795 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.057975 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.058655 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.059391 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.060555 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.060984 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.061115 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.064535 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.064549 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.064561 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.064590 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.066771 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.067528 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.067535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.067536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.067633 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.068030 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.068036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.068043 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.068358 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.068529 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.068997 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.069207 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":127,"threads_started":1}
I20260811 22:19:01.070652 1496 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fea480280f4ac9b3e05d83696af4b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:00 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:58860
I20260811 22:19:01.071657 1585 tablet_replica.cc:330] stopping tablet replica
I20260811 22:19:01.072042 1585 raft_consensus.cc:2229] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:01.072335 1585 raft_consensus.cc:2258] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:01.073537 1585 ts_tablet_manager.cc:1905] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:01.077996 1585 ts_tablet_manager.cc:1918] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:01.078115 1585 log.cc:1196] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/20fea480280f4ac9b3e05d83696af4b9
I20260811 22:19:01.078449 1585 ts_tablet_manager.cc:1939] T 20fea480280f4ac9b3e05d83696af4b9 P 67dbc860a99d4115938224c8b9f0846f: Deleting consensus metadata
I20260811 22:19:01.080492 1337 catalog_manager.cc:4841] TS 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823): tablet 20fea480280f4ac9b3e05d83696af4b9 (table test-table [id=74fe4b25e2b744f9807df60f8a23f358]) successfully deleted
W20260811 22:19:01.161185 1579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:01.161549 1579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:01.161603 1579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:01.166042 1579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:01.166131 1579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:01.166198 1579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:19:01.171976 1579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:43039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=46461
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:01.173197 1579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:01.174347 1579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:01.185334 1592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.185314 1591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.185375 1594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:01.185909 1579 server_base.cc:1034] running on GCE node
I20260811 22:19:01.186461 1579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:01.187189 1579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:01.188400 1579 hybrid_clock.cc:648] HybridClock initialized: now 1786486741188370 us; error 46 us; skew 500 ppm
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.191926 1579 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:19:01.193061 1579 webserver.cc:460] Webserver started at http://127.17.198.2:46461/ using document root <none> and password file <none>
I20260811 22:19:01.193703 1579 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:01.193809 1579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:01.198400 1579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:01.200953 1603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.202521 1579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:01.202708 1579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "e889c3b6dcad43088aee18d0d92399b8"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "62cb9730c758c8c288d002a2eb66845e"
server_key_iv: "abd3368a55ec62dab862075fa924bd75"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.203379 1579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:01.225800 1579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:01.229982 1579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:01.230278 1579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:01.231026 1579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:01.232102 1579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:01.232190 1579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.232278 1579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:01.232331 1579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.243136 1579 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43039
I20260811 22:19:01.243186 1730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43039 every 8 connection(s)
I20260811 22:19:01.244405 1579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:01.247385 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1579
I20260811 22:19:01.247975 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:35803
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=46655
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:01.262377 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.246743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:41587 (local address 127.17.198.62:35391)
0811 22:19:01.246951 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.246957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.248123 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 22:19:01.249958 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.249962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.250036 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.250163 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.251682 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.252296 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.253878 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.254168 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.254315 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.257546 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.257550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.257553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.257586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.259359 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.260339 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.260345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.260347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.260472 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.260962 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.260967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.260969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.261228 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.261336 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.262020 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.262198 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:19:01.263253 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.247190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:41587)
0811 22:19:01.247899 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.247969 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.249675 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.250387 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.250395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.250836 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:01.251490 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.251508 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.252478 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.252481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.253714 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.253727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.254376 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.254380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.254604 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.255390 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:01.255411 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:01.257276 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:01.259535 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.259543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.259565 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.260024 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.260601 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.260604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.260605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.260786 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.261341 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:01.261347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:01.261661 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.261842 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.262176 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":184,"threads_started":1}
I20260811 22:19:01.264714 1731 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
I20260811 22:19:01.265065 1731 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:01.265697 1731 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
I20260811 22:19:01.267004 1353 ts_manager.cc:194] Registered new tserver with Master: e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039)
I20260811 22:19:01.267743 1353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:41587
W20260811 22:19:01.375554 1734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:01.375888 1734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:01.375938 1734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:01.380288 1734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:01.380401 1734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:01.380468 1734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:19:01.386584 1734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:35803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=46655
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:01.387861 1734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:01.389041 1734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:01.400251 1741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.400234 1743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.400234 1740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:01.400288 1734 server_base.cc:1034] running on GCE node
I20260811 22:19:01.401060 1734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:01.401810 1734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:01.403007 1734 hybrid_clock.cc:648] HybridClock initialized: now 1786486741402989 us; error 32 us; skew 500 ppm
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.406602 1734 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:19:01.407764 1734 webserver.cc:460] Webserver started at http://127.17.198.3:46655/ using document root <none> and password file <none>
I20260811 22:19:01.408494 1734 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:01.408605 1734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:01.413137 1734 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:01.415854 1751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.417481 1734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:01.417665 1734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "e679a73a13244e5c881d28b77b69859a"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-77v9"
server_key: "a09b1c2a2bc2f20a2bfc9257e4260d1c"
server_key_iv: "0d07e6e5119fafd07962262877621b97"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.418298 1734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:01.439590 1734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:01.443751 1734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:01.444018 1734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:01.444738 1734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:01.445755 1734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:01.445847 1734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.445945 1734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:01.445986 1734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.456781 1734 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:35803
I20260811 22:19:01.456799 1882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:35803 every 8 connection(s)
I20260811 22:19:01.458002 1734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:01.466987 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1734
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:01.475420 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.460160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:46079 (local address 127.17.198.62:35391)
0811 22:19:01.460339 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.460343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.461275 (+ 932us) server_negotiation.cc:407] Connection header received
0811 22:19:01.462922 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.462931 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.463000 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.463098 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.464636 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.465434 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.466915 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.467201 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.467329 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.470760 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.470763 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.470765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.470793 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.472458 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.473142 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.473145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.473148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.473238 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.473630 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.473633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.473635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.473984 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.474105 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.475054 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.475204 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:19:01.476504 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.460565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:46079)
0811 22:19:01.461098 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.461146 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.462570 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.463333 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.463342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.463780 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:01.464427 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.464443 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.465583 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.465586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.466721 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.466734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.467406 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.467411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.467700 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.468476 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:01.468503 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:01.470556 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:01.472596 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.472604 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.472624 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.473030 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.473340 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.473344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.473354 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.473520 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.474130 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:01.474136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:01.474625 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.474947 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.475372 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":184,"threads_started":1}
I20260811 22:19:01.477699 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.468000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.17.198.62:35391)
0811 22:19:01.468328 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.468334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.468395 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:19:01.468584 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.468587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.468633 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.468711 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.469726 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.470365 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.471249 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.471521 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.471640 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.473233 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.473239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.473248 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.473273 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.475167 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.475757 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.475764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.475767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.475857 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.476249 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.476252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.476253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.476465 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.476688 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.477423 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.477569 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
I20260811 22:19:01.478047 1883 heartbeater.cc:347] Connected to a master server at 127.17.198.62:35391
I20260811 22:19:01.478375 1883 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:01.479030 1883 heartbeater.cc:511] Master 127.17.198.62:35391 requested a full tablet report, sending...
I20260811 22:19:01.480151 1353 ts_manager.cc:194] Registered new tserver with Master: e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
I20260811 22:19:01.480743 1353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:46079
I20260811 22:19:01.492167 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:19:01.497828 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.492965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.17.198.62:35391)
0811 22:19:01.493102 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.493106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.493395 (+ 289us) server_negotiation.cc:407] Connection header received
0811 22:19:01.493525 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.493527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.493647 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.493835 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:01.494123 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.494722 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.495704 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.495975 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.496065 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.496610 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:01.497280 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:01.497404 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.497563 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.497639 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:19:01.498606 1353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:01.502081 1353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:01.512140 1496 tablet_service.cc:1467] Processing CreateTablet for tablet eee68c3326a44972ad1a18b985b8ed5d (DEFAULT_TABLE table=test-table [id=b12eadd4000248569d8f391671551b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:01.513031 1496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee68c3326a44972ad1a18b985b8ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:01.516551 1447 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Bootstrap starting.
I20260811 22:19:01.517624 1447 tablet_bootstrap.cc:654] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:01.517714 1583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.510912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.0.0.1:36188)
0811 22:19:01.511109 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.511127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.511224 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.512556 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.512560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.512579 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.512721 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.512728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.514473 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.514476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.515613 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.515624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.517212 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.517217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.517322 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.517384 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.517479 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.517568 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 22:19:01.518404 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.511419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.17.198.2:43039)
0811 22:19:01.511909 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.511923 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.511950 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:19:01.512051 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.512057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.512326 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.512940 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.513132 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.514287 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.515927 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.517079 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.517201 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.517363 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.518128 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.518245 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":167,"threads_started":1}
I20260811 22:19:01.519330 1447 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: No bootstrap required, opened a new log
I20260811 22:19:01.519433 1447 ts_tablet_manager.cc:1397] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:01.519726 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.511132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:35803 (local address 127.0.0.1:51366)
0811 22:19:01.511496 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.511512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.511614 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.515266 (+ 3652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.515270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.515295 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.515394 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.515400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.516903 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.516907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.517958 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.517969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.519285 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.519289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.519384 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.519412 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.519499 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.519590 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":125,"threads_started":1}
I20260811 22:19:01.519816 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.511314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.17.198.3:35803)
0811 22:19:01.514668 (+ 3354us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.514681 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.514709 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:19:01.514798 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.514804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.515075 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.515279 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.515562 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.516715 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.518105 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.518969 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.519077 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.519218 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.519558 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.519675 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3215,"thread_start_us":178,"threads_started":1}
I20260811 22:19:01.519994 1447 raft_consensus.cc:348] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.520124 1447 raft_consensus.cc:374] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:01.520166 1447 raft_consensus.cc:729] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dbc860a99d4115938224c8b9f0846f, State: Initialized, Role: FOLLOWER
I20260811 22:19:01.520566 1447 consensus_queue.cc:260] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.521529 1447 ts_tablet_manager.cc:1428] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:19:01.522128 1657 tablet_service.cc:1467] Processing CreateTablet for tablet eee68c3326a44972ad1a18b985b8ed5d (DEFAULT_TABLE table=test-table [id=b12eadd4000248569d8f391671551b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:01.522745 1806 tablet_service.cc:1467] Processing CreateTablet for tablet eee68c3326a44972ad1a18b985b8ed5d (DEFAULT_TABLE table=test-table [id=b12eadd4000248569d8f391671551b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:01.523411 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee68c3326a44972ad1a18b985b8ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:01.523801 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee68c3326a44972ad1a18b985b8ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 22:19:01.524998 1576 tablet.cc:2369] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:01.532582 1898 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Bootstrap starting.
I20260811 22:19:01.533226 1897 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Bootstrap starting.
I20260811 22:19:01.535629 1898 tablet_bootstrap.cc:654] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:01.536157 1897 tablet_bootstrap.cc:654] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:01.537420 1898 log.cc:824] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:01.537420 1897 log.cc:824] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:01.540104 1898 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: No bootstrap required, opened a new log
I20260811 22:19:01.540104 1897 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: No bootstrap required, opened a new log
I20260811 22:19:01.540359 1898 ts_tablet_manager.cc:1397] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:19:01.540359 1897 ts_tablet_manager.cc:1397] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260811 22:19:01.544924 1898 raft_consensus.cc:348] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.544924 1897 raft_consensus.cc:348] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.545452 1897 raft_consensus.cc:374] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:01.545451 1898 raft_consensus.cc:374] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:01.545526 1897 raft_consensus.cc:729] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e889c3b6dcad43088aee18d0d92399b8, State: Initialized, Role: FOLLOWER
I20260811 22:19:01.545538 1898 raft_consensus.cc:729] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e679a73a13244e5c881d28b77b69859a, State: Initialized, Role: FOLLOWER
I20260811 22:19:01.546135 1897 consensus_queue.cc:260] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.546135 1898 consensus_queue.cc:260] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.547099 1883 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
I20260811 22:19:01.547336 1731 heartbeater.cc:503] Master 127.17.198.62:35391 was elected leader, sending a full tablet report...
I20260811 22:19:01.547603 1898 ts_tablet_manager.cc:1428] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:19:01.547603 1897 ts_tablet_manager.cc:1428] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 22:19:01.593137 1581 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:01.593464 1581 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.594732 1581 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
I20260811 22:19:01.595534 1902 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:01.595742 1902 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.597191 1902 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823)
I20260811 22:19:01.600448 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.595164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:41053 (local address 127.17.198.2:43039)
0811 22:19:01.595469 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.595474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.595727 (+ 253us) server_negotiation.cc:407] Connection header received
0811 22:19:01.596092 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.596096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.596175 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.596269 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.596628 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.597462 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.599138 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.599705 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.599854 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.599918 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.600183 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.600279 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":210}
I20260811 22:19:01.600543 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.595269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:48637 (local address 127.17.198.3:35803)
0811 22:19:01.595408 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.595414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.596164 (+ 750us) server_negotiation.cc:407] Connection header received
0811 22:19:01.596264 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.596267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.596312 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.596390 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.596727 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.597444 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.599213 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.599891 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.600003 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.600047 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.600340 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.600415 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":54}
I20260811 22:19:01.600936 1903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.595119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:35803 (local address 127.17.198.1:48637)
0811 22:19:01.596074 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.596088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.596179 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.596444 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.596447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.596480 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.596596 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.596605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.597590 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.597594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.599066 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.599078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.600110 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.600116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.600207 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.600229 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.600707 (+ 478us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.600764 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":70,"thread_start_us":83,"threads_started":1}
I20260811 22:19:01.600939 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.595064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.17.198.1:41053)
0811 22:19:01.595624 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.595642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.595833 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.596334 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.596337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.596354 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.596490 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.596497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.597612 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.597615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.598997 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.599011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.599851 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.599855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.599983 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.600053 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.600622 (+ 569us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.600727 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":46,"thread_start_us":116,"threads_started":1}
I20260811 22:19:01.601671 1828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "67dbc860a99d4115938224c8b9f0846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e679a73a13244e5c881d28b77b69859a" is_pre_election: true
I20260811 22:19:01.601890 1679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "67dbc860a99d4115938224c8b9f0846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e889c3b6dcad43088aee18d0d92399b8" is_pre_election: true
I20260811 22:19:01.601943 1828 raft_consensus.cc:2454] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67dbc860a99d4115938224c8b9f0846f in term 0.
I20260811 22:19:01.602195 1679 raft_consensus.cc:2454] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67dbc860a99d4115938224c8b9f0846f in term 0.
I20260811 22:19:01.602427 1441 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f, e679a73a13244e5c881d28b77b69859a; no voters:
I20260811 22:19:01.602726 1581 raft_consensus.cc:2781] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:01.602844 1581 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:01.602900 1581 raft_consensus.cc:3037] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:01.603982 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.597431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.17.198.3:39159)
0811 22:19:01.597802 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.597817 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.597907 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.598691 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.598695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.598714 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.598840 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.598854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.601364 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.601368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.602577 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.602586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.603521 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.603525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.603642 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.603669 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.603741 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.603848 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":10}
I20260811 22:19:01.604158 1905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.597594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.17.198.3:59745)
0811 22:19:01.597986 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.598001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.598095 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.599673 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.599676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.599697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:01.599886 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.599895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.601015 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.601020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.602778 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.602788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.603661 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.603665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.603761 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.603784 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.603877 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.603953 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":480,"thread_start_us":125,"threads_started":1}
I20260811 22:19:01.604270 1907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.598431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:59745 (local address 127.17.198.1:41823)
0811 22:19:01.599446 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.599450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.599463 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:19:01.599502 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.599504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.599545 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.599632 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.600021 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.600879 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.603034 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.603540 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.603666 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.603798 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.603895 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.603972 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":73,"threads_started":1}
I20260811 22:19:01.604488 1581 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.604833 1906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.598082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:39159 (local address 127.17.198.2:43039)
0811 22:19:01.598450 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.598458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.598472 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:19:01.598516 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.598519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.598561 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.598958 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:01.599094 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.600951 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.602712 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.603371 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.603497 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.603545 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.604577 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.604677 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":301,"thread_start_us":133,"threads_started":1}
I20260811 22:19:01.604959 1581 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 1 election: Requested vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
I20260811 22:19:01.605275 1522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67dbc860a99d4115938224c8b9f0846f" is_pre_election: true
I20260811 22:19:01.605355 1679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "e679a73a13244e5c881d28b77b69859a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e889c3b6dcad43088aee18d0d92399b8" is_pre_election: true
I20260811 22:19:01.605367 1678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "67dbc860a99d4115938224c8b9f0846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e889c3b6dcad43088aee18d0d92399b8"
I20260811 22:19:01.605517 1679 raft_consensus.cc:2454] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e679a73a13244e5c881d28b77b69859a in term 0.
I20260811 22:19:01.605569 1522 raft_consensus.cc:2379] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e679a73a13244e5c881d28b77b69859a in current term 1: Already voted for candidate 67dbc860a99d4115938224c8b9f0846f in this term.
I20260811 22:19:01.605603 1828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee68c3326a44972ad1a18b985b8ed5d" candidate_uuid: "67dbc860a99d4115938224c8b9f0846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e679a73a13244e5c881d28b77b69859a"
I20260811 22:19:01.605738 1828 raft_consensus.cc:3037] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:01.605959 1752 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8; no voters:
I20260811 22:19:01.607154 1828 raft_consensus.cc:2454] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67dbc860a99d4115938224c8b9f0846f in term 1.
I20260811 22:19:01.607295 1902 raft_consensus.cc:2741] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 22:19:01.607546 1441 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f, e679a73a13244e5c881d28b77b69859a; no voters: e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:01.607752 1581 raft_consensus.cc:2781] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:01.608011 1581 raft_consensus.cc:686] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 LEADER]: Becoming Leader. State: Replica: 67dbc860a99d4115938224c8b9f0846f, State: Running, Role: LEADER
I20260811 22:19:01.608333 1581 consensus_queue.cc:237] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:01.611726 1353 catalog_manager.cc:5495] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f reported cstate change: term changed from 0 to 1, leader changed from <none> to 67dbc860a99d4115938224c8b9f0846f (127.17.198.1). New cstate: current_term: 1 leader_uuid: "67dbc860a99d4115938224c8b9f0846f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } health_report { overall_health: HEALTHY } } }
I20260811 22:19:01.629506 1907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.625519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.17.198.1:41823)
0811 22:19:01.625680 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.625684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.625701 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:19:01.625863 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.625866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.625941 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.626081 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:01.626393 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.626914 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.627848 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.628074 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.628190 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.628516 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:01.629077 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:01.629214 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.629275 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.629342 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:19:01.633790 1678 raft_consensus.cc:3037] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:01.633984 1828 raft_consensus.cc:1261] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Refusing update from remote peer 67dbc860a99d4115938224c8b9f0846f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:01.634768 1908 consensus_queue.cc:1035] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:01.635550 1678 raft_consensus.cc:1261] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Refusing update from remote peer 67dbc860a99d4115938224c8b9f0846f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:01.636266 1908 consensus_queue.cc:1035] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:01.647171 1913 mvcc.cc:205] Tried to move back new op lower bound from 7317449693725388800 to 7317449693630803968. Current Snapshot: MvccSnapshot[applied={T|T < 7317449693725388800 or (T in {7317449693725388800})}]
I20260811 22:19:01.653477 1353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:01.653668 1353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:01.656598 1353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Sending DeleteTablet for 3 replicas of tablet eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:01.657560 1657 tablet_service.cc:1514] Processing DeleteTablet for tablet eee68c3326a44972ad1a18b985b8ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:01 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36188
I20260811 22:19:01.657789 1806 tablet_service.cc:1514] Processing DeleteTablet for tablet eee68c3326a44972ad1a18b985b8ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:01 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:51366
I20260811 22:19:01.658417 1925 tablet_replica.cc:330] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: stopping tablet replica
I20260811 22:19:01.658483 1496 tablet_service.cc:1514] Processing DeleteTablet for tablet eee68c3326a44972ad1a18b985b8ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:01 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:58860
I20260811 22:19:01.658802 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 1398
I20260811 22:19:01.661690 1925 raft_consensus.cc:2229] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:01.662202 1925 raft_consensus.cc:2258] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:01.663612 1924 tablet_replica.cc:330] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: stopping tablet replica
I20260811 22:19:01.664196 1925 ts_tablet_manager.cc:1905] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:01.664251 1924 raft_consensus.cc:2229] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:01.664723 1924 raft_consensus.cc:2258] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:01.667847 1925 ts_tablet_manager.cc:1918] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:01.667975 1925 log.cc:1196] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:01.668694 1925 ts_tablet_manager.cc:1939] T eee68c3326a44972ad1a18b985b8ed5d P e679a73a13244e5c881d28b77b69859a: Deleting consensus metadata
I20260811 22:19:01.669534 1924 ts_tablet_manager.cc:1905] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:01.670295 1335 catalog_manager.cc:4841] TS e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): tablet eee68c3326a44972ad1a18b985b8ed5d (table test-table [id=b12eadd4000248569d8f391671551b64]) successfully deleted
I20260811 22:19:01.673831 1924 ts_tablet_manager.cc:1918] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:01.673990 1924 log.cc:1196] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:01.674351 1924 ts_tablet_manager.cc:1939] T eee68c3326a44972ad1a18b985b8ed5d P e889c3b6dcad43088aee18d0d92399b8: Deleting consensus metadata
I20260811 22:19:01.675823 1334 catalog_manager.cc:4841] TS e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): tablet eee68c3326a44972ad1a18b985b8ed5d (table test-table [id=b12eadd4000248569d8f391671551b64]) successfully deleted
W20260811 22:19:01.680402 1337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:41823 (error 108)
I20260811 22:19:01.680769 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 1579
I20260811 22:19:01.687911 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.687468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:41823 (local address 127.0.0.1:58872)
0811 22:19:01.687652 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.687761 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 22:19:01.688359 1337 catalog_manager.cc:4584] TS 67dbc860a99d4115938224c8b9f0846f (127.17.198.1:41823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eee68c3326a44972ad1a18b985b8ed5d: Network error: Client connection negotiation failed: client connection to 127.17.198.1:41823: connect: Connection refused (error 111)
I20260811 22:19:01.693608 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 1734
I20260811 22:19:01.703842 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 1312
I20260811 22:19:01.714241 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--webserver_interface=127.17.198.62
--webserver_port=37843
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:35391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:01.845413 1929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:01.845738 1929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:01.845784 1929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:01.850191 1929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:01.850284 1929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:01.850307 1929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:01.850327 1929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:01.850343 1929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:01.856515 1929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:35391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:35391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=37843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:01.857798 1929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:01.858881 1929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:01.868126 1934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.868378 1935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.868666 1937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:01.868695 1929 server_base.cc:1034] running on GCE node
I20260811 22:19:01.869604 1929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:01.870939 1929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:01.872134 1929 hybrid_clock.cc:648] HybridClock initialized: now 1786486741872091 us; error 60 us; skew 500 ppm
Aug 11 22:19:01 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.875563 1929 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:19:01.876720 1929 webserver.cc:460] Webserver started at http://127.17.198.62:37843/ using document root <none> and password file <none>
I20260811 22:19:01.877413 1929 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:01.877493 1929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:01.882133 1929 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:01.884608 1945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.886229 1929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:19:01.886546 1929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "8a9e110b184542c28c5dad6936b2ba94"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "e3b251f3d21715548248fc3078de190f"
server_key_iv: "14420d1bd08892ca381ebdf0303376d1"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.887219 1929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:01.909013 1929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:01.913055 1929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:01.913355 1929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:01.922286 1929 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:35391
I20260811 22:19:01.922612 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:35391 every 8 connection(s)
I20260811 22:19:01.923691 1929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:01.928583 2004 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:01.930570 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap starting.
I20260811 22:19:01.933676 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 1929
I20260811 22:19:01.934249 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:41823
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=45295
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:01.950939 2004 log.cc:824] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:01.963830 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:01.964351 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94: Bootstrap complete.
I20260811 22:19:01.968793 2004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:01.969359 2004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Initialized, Role: FOLLOWER
I20260811 22:19:01.969980 2004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:01.970109 2004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:01.970203 2004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:01.970333 2004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:19:01.972466 2004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:01.972904 2004 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9e110b184542c28c5dad6936b2ba94; no voters:
I20260811 22:19:01.973308 2004 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:19:01.973412 2008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:19:01.973649 2008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [term 3 LEADER]: Becoming Leader. State: Replica: 8a9e110b184542c28c5dad6936b2ba94, State: Running, Role: LEADER
I20260811 22:19:01.973959 2008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } }
I20260811 22:19:01.974644 2004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:01.975023 2008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9e110b184542c28c5dad6936b2ba94. Latest consensus state: current_term: 3 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:19:01.975152 2008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:01.975531 2009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8a9e110b184542c28c5dad6936b2ba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9e110b184542c28c5dad6936b2ba94" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 35391 } } }
I20260811 22:19:01.975759 2009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9e110b184542c28c5dad6936b2ba94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:01.975550 2014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:01.981591 2014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:01.982586 2014 catalog_manager.cc:1259] Loaded cluster ID: 143a0639f73745fd991b3fc2f2c330dc
I20260811 22:19:01.982672 2014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:19:01.983830 2014 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7feb022089ad google::LogMessage::Fail() at ??:0
@ 0x7feb0220b88a google::LogMessage::SendToLog() at ??:0
@ 0x7feb022084de google::LogMessage::Flush() at ??:0
@ 0x7feb02209a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7feb0cb47fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7feb0cb48d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7feb0cb45579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7feb0cb89640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7feb0445a3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7feb0445ac83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7feb0445c3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7feb0444b512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7feb05a8f6db start_thread at ??:0
@ 0x7feb00d0661f clone at ??:0
W20260811 22:19:02.095969 2006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:02.096284 2006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:02.096333 2006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:02.101397 2006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:02.101483 2006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:02.101577 2006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:19:02.107406 2006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:41823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=45295
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:02.108686 2006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:02.109936 2006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:02.120898 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:02.120973 2006 server_base.cc:1034] running on GCE node
W20260811 22:19:02.120900 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:02.121299 2053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:02.121825 2006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:02.122670 2006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:02.123920 2006 hybrid_clock.cc:648] HybridClock initialized: now 1786486742123878 us; error 65 us; skew 500 ppm
Aug 11 22:19:02 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:02.127532 2006 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:19:02.128800 2006 webserver.cc:460] Webserver started at http://127.17.198.1:45295/ using document root <none> and password file <none>
I20260811 22:19:02.129590 2006 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:02.129714 2006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:02.134562 2006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:19:02.137235 2063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.138872 2006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:19:02.139051 2006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "67dbc860a99d4115938224c8b9f0846f"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "0daa1b3cefc24dd43b50081edebb7731"
server_key_iv: "1c6592a6216e4867f714c0ebf484da16"
server_key_version: "encryptionkey@0"
I20260811 22:19:02.139720 2006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:02.161198 2006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:02.165637 2006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:02.165892 2006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:02.166796 2006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:02.168177 2071 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:19:02.174396 2006 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:19:02.174535 2006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 22:19:02.174631 2006 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:19:02.176730 2006 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:19:02.176915 2006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.003s sys 0.000s
I20260811 22:19:02.177392 2071 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Bootstrap starting.
I20260811 22:19:02.188963 2006 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:41823
I20260811 22:19:02.189594 2191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:41823 every 8 connection(s)
I20260811 22:19:02.190359 2006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:02.194728 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2006
I20260811 22:19:02.195333 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:43039
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=46461
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:02.201499 2071 log.cc:824] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:02.212008 2071 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:02.212539 2071 tablet_bootstrap.cc:492] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Bootstrap complete.
I20260811 22:19:02.214188 2071 ts_tablet_manager.cc:1397] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Time spent bootstrapping tablet: real 0.037s user 0.028s sys 0.007s
I20260811 22:19:02.219563 2071 raft_consensus.cc:348] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:02.220191 2071 raft_consensus.cc:729] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dbc860a99d4115938224c8b9f0846f, State: Initialized, Role: FOLLOWER
I20260811 22:19:02.220767 2071 consensus_queue.cc:260] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:02.221935 2071 ts_tablet_manager.cc:1428] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260811 22:19:02.223644 2194 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:02.193023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:49997)
0811 22:19:02.195389 (+ 2366us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.195728 (+ 339us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:02.197250 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:02.222659 (+ 25409us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2049,"thread_start_us":151,"threads_started":1}
W20260811 22:19:02.224346 2192 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:35391 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:35391: Client connection negotiation failed: client connection to 127.17.198.62:35391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:19:02.225306 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.224985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:56253)
0811 22:19:02.225134 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.225224 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:19:02.226156 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.225899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:50205)
0811 22:19:02.226036 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.226099 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:19:02.226454 2192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:19:02.322839 2195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:02.323160 2195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:02.323207 2195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:02.327479 2195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:02.327577 2195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:02.327641 2195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:19:02.334263 2195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:43039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=46461
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:02.335532 2195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:02.336644 2195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:02.348026 2209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:02.348026 2206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:02.348026 2205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:02.348521 2195 server_base.cc:1034] running on GCE node
I20260811 22:19:02.349123 2195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:02.349921 2195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:02.351162 2195 hybrid_clock.cc:648] HybridClock initialized: now 1786486742351119 us; error 60 us; skew 500 ppm
Aug 11 22:19:02 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:02.354615 2195 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:19:02.355837 2195 webserver.cc:460] Webserver started at http://127.17.198.2:46461/ using document root <none> and password file <none>
I20260811 22:19:02.356472 2195 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:02.356547 2195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:02.361445 2195 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:02.364229 2217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.365782 2195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:02.365988 2195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "e889c3b6dcad43088aee18d0d92399b8"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-77v9"
server_key: "62cb9730c758c8c288d002a2eb66845e"
server_key_iv: "abd3368a55ec62dab862075fa924bd75"
server_key_version: "encryptionkey@0"
I20260811 22:19:02.366628 2195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:02.391387 2195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:02.395136 2195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:02.395324 2195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:02.395942 2195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:02.396831 2195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:02.396893 2195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.396936 2195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:02.396951 2195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.407596 2195 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:43039
I20260811 22:19:02.407697 2345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:43039 every 8 connection(s)
I20260811 22:19:02.408834 2195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:02.413653 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.411363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:36073)
0811 22:19:02.411987 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.412129 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":126,"threads_started":1}
W20260811 22:19:02.414484 2347 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:35391 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:35391: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
I20260811 22:19:02.415345 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.415021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:46667)
0811 22:19:02.415192 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.415264 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 22:19:02.415509 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2195
I20260811 22:19:02.416050 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:35803
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:35391
--webserver_port=46655
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:02.416102 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.415785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:52927)
0811 22:19:02.415982 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.416035 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 22:19:02.416318 2347 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:19:02.541899 2350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:02.542214 2350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:02.542299 2350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:02.546736 2350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:02.546818 2350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:02.546885 2350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:19:02.552534 2350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:35803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=46655
--tserver_master_addrs=127.17.198.62:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:02.553792 2350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:02.554941 2350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:02.565799 2359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:02.565824 2357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:02.565816 2356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:02.566305 2350 server_base.cc:1034] running on GCE node
I20260811 22:19:02.566840 2350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:02.567643 2350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:02.568945 2350 hybrid_clock.cc:648] HybridClock initialized: now 1786486742568906 us; error 54 us; skew 500 ppm
Aug 11 22:19:02 dist-test-slave-77v9 krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:02.572646 2350 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:19:02.573853 2350 webserver.cc:460] Webserver started at http://127.17.198.3:46655/ using document root <none> and password file <none>
I20260811 22:19:02.574563 2350 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:02.574681 2350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:02.579438 2350 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:02.582068 2366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.583647 2350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:02.583837 2350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "e679a73a13244e5c881d28b77b69859a"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-77v9"
server_key: "a09b1c2a2bc2f20a2bfc9257e4260d1c"
server_key_iv: "0d07e6e5119fafd07962262877621b97"
server_key_version: "encryptionkey@0"
I20260811 22:19:02.584537 2350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:02.599628 2350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:02.603695 2350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:02.603958 2350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:02.604679 2350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:02.605746 2350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:02.605842 2350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.605940 2350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:02.606002 2350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:02.619561 2350 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:35803
I20260811 22:19:02.619648 2496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:35803 every 8 connection(s)
I20260811 22:19:02.620857 2350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:02.625734 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2350
I20260811 22:19:02.625952 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.623362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:37879)
0811 22:19:02.624126 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.624290 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":134,"threads_started":1}
W20260811 22:19:02.626600 2497 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:35391 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:35391: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
I20260811 22:19:02.627305 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.627016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:33469)
0811 22:19:02.627158 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.627218 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:19:02.628082 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:02.627828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:33967)
0811 22:19:02.627962 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:02.628017 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:19:02.628305 2497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:19:03.227658 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.227167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:48957)
0811 22:19:03.227492 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:03.227555 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":91,"threads_started":1}
I20260811 22:19:03.417781 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.417215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:49833)
0811 22:19:03.417544 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:03.417636 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260811 22:19:03.629756 2504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.629188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:48991)
0811 22:19:03.629545 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:03.629621 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
I20260811 22:19:03.749380 2505 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:03.749639 2505 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:03.750819 2505 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
Aug 11 22:19:03 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.3@KRBTEST.COM
Aug 11 22:19:03 dist-test-slave-77v9 krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.2@KRBTEST.COM
I20260811 22:19:03.767802 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.751658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:39237 (local address 127.17.198.2:43039)
0811 22:19:03.751990 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:03.752013 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:03.752046 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:19:03.752728 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:03.752734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:03.753147 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:03.753479 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:03.755087 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.756171 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.757416 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.758078 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.758283 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:03.762094 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:03.762099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:03.762111 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:03.762144 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:03.764261 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765008 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:03.765013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:03.765018 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:03.765161 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765948 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:03.765952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:03.765953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:03.766496 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:03.766656 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:03.767226 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 22:19:03.767581 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":117,"threads_started":1}
I20260811 22:19:03.767799 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.751351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38279 (local address 127.17.198.3:35803)
0811 22:19:03.751836 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:03.751872 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:03.751923 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:19:03.752656 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:03.752662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:03.753117 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:03.753449 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:03.755001 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.756082 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.757419 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.758006 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.758201 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:03.761917 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:03.761922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:03.761937 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:03.761973 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:03.764208 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765003 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:03.765009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:03.765014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:03.765149 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765762 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:03.765765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:03.765767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:03.766460 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:03.766640 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:03.767236 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 22:19:03.767563 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":94,"threads_started":1}
I20260811 22:19:03.767854 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.751179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:43039 (local address 127.17.198.1:39237)
0811 22:19:03.751647 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:03.751661 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:03.751751 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:03.753567 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:03.753573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:03.754198 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:03.754781 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:03.754806 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.756335 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.756343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:03.757272 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:03.757283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.758224 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.758228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:03.758362 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:03.759198 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:03.759212 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:03.761856 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:03.764424 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:03.764429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:03.764431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:03.764856 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:03.765727 (+ 871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:03.765731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:03.765823 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:03.766659 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:03.766665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:03.766891 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 22:19:03.767087 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:19:03.767563 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":319,"spinlock_wait_cycles":3840,"thread_start_us":79,"threads_started":1}
I20260811 22:19:03.768203 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:03.751179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:35803 (local address 127.17.198.1:38279)
0811 22:19:03.751541 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:03.751561 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:03.751689 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:03.753567 (+ 1878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:03.753573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:03.754209 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:03.754783 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:03.754808 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.756245 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.756249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:03.757270 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:03.757281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:03.758149 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:03.758153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:03.758353 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:03.759149 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:03.759167 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:03.761488 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:03.764381 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:03.764388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:03.764405 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:03.764854 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:03.765352 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:03.765355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:03.765357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:03.765594 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:03.766618 (+ 1024us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:03.766625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:03.766878 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 22:19:03.767085 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:19:03.767650 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":288,"spinlock_wait_cycles":3584,"thread_start_us":85,"threads_started":1}
W20260811 22:19:03.770866 2064 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
W20260811 22:19:03.771032 2066 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:03.771107 2064 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f; no voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:03.771584 2505 raft_consensus.cc:2726] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:04.229168 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:04.228771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:37063)
0811 22:19:04.228987 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:04.229063 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 22:19:04.419063 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:04.418624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:41693)
0811 22:19:04.418910 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:04.418968 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":78,"threads_started":1}
I20260811 22:19:04.631711 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:04.631061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:35337)
0811 22:19:04.631498 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:04.631578 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":109,"threads_started":1}
I20260811 22:19:05.230495 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.230018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:37025)
0811 22:19:05.230307 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:05.230383 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":121,"threads_started":1}
I20260811 22:19:05.420710 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.420245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:59147)
0811 22:19:05.420544 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:05.420612 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
I20260811 22:19:05.453171 2515 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:05.453334 2515 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:05.453938 2515 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
W20260811 22:19:05.454895 2064 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
W20260811 22:19:05.455001 2066 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:05.455132 2066 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f; no voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:05.455279 2515 raft_consensus.cc:2726] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:05.633490 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.632957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:60225)
0811 22:19:05.633274 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:05.633352 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":80,"threads_started":1}
I20260811 22:19:06.232261 2517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.231591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:54477)
0811 22:19:06.232067 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.232144 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":105,"threads_started":1}
I20260811 22:19:06.422374 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.421743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:37413)
0811 22:19:06.422188 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.422267 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":263,"threads_started":1}
I20260811 22:19:06.635131 2519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.634583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:47327)
0811 22:19:06.634935 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.635023 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260811 22:19:07.186863 2520 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:07.187018 2520 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:07.187448 2520 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
W20260811 22:19:07.188438 2064 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
W20260811 22:19:07.188638 2066 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:07.188755 2066 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f; no voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:07.188985 2520 raft_consensus.cc:2726] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:07.234233 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.233613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:44449)
0811 22:19:07.234047 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.234123 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":87,"threads_started":1}
I20260811 22:19:07.424064 2522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.423490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:40689)
0811 22:19:07.423843 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.423914 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":95,"threads_started":1}
I20260811 22:19:07.636685 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.636168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:52379)
0811 22:19:07.636517 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.636584 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260811 22:19:08.235860 2524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.235253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:34723)
0811 22:19:08.235621 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.235695 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260811 22:19:08.425609 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.425150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:56157)
0811 22:19:08.425443 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.425508 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
I20260811 22:19:08.638185 2526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.637688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:51549)
0811 22:19:08.638017 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.638076 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260811 22:19:09.237555 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.236990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:59299)
0811 22:19:09.237375 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.237444 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260811 22:19:09.309444 2528 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:09.309623 2528 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:09.310175 2528 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
W20260811 22:19:09.311053 2064 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
W20260811 22:19:09.311209 2066 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:09.311347 2066 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f; no voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:09.311514 2528 raft_consensus.cc:2726] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:09.427340 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.426694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:49979)
0811 22:19:09.427091 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.427177 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":78,"threads_started":1}
I20260811 22:19:09.639777 2530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.639211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:51497)
0811 22:19:09.639586 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.639664 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":89,"threads_started":1}
I20260811 22:19:10.238965 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.238457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:49513)
0811 22:19:10.238744 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.238810 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260811 22:19:10.428771 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.428297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:38361)
0811 22:19:10.428600 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.428669 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
I20260811 22:19:10.641433 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.640964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:45335)
0811 22:19:10.641256 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.641326 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260811 22:19:11.240574 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.240027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:40163)
0811 22:19:11.240400 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.240473 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":96,"threads_started":1}
I20260811 22:19:11.430460 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.429909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:38819)
0811 22:19:11.430208 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.430295 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260811 22:19:11.642938 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.642442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:43891)
0811 22:19:11.642760 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.642837 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":90,"threads_started":1}
I20260811 22:19:12.242314 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.241833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:37299)
0811 22:19:12.242144 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.242212 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":88,"threads_started":1}
I20260811 22:19:12.432008 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.431454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:45451)
0811 22:19:12.431795 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.431877 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260811 22:19:12.644546 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.644037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:33053)
0811 22:19:12.644354 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.644444 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260811 22:19:13.244099 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:13.243449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:59481)
0811 22:19:13.243910 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:13.243987 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":118,"threads_started":1}
I20260811 22:19:13.433701 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:13.433042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:47991)
0811 22:19:13.433455 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:13.433541 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":95,"threads_started":1}
I20260811 22:19:13.646103 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:13.645597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:49979)
0811 22:19:13.645915 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:13.645998 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
I20260811 22:19:14.245532 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:14.245077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:43541)
0811 22:19:14.245360 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:14.245427 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260811 22:19:14.435168 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:14.434653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:33585)
0811 22:19:14.434993 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:14.435062 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
I20260811 22:19:14.647644 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:14.647094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:33821)
0811 22:19:14.647432 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:14.647513 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
I20260811 22:19:14.889319 2548 raft_consensus.cc:482] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:14.889494 2548 raft_consensus.cc:504] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e889c3b6dcad43088aee18d0d92399b8" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 43039 } } peers { permanent_uuid: "e679a73a13244e5c881d28b77b69859a" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 35803 } } peers { permanent_uuid: "67dbc860a99d4115938224c8b9f0846f" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 41823 } }
I20260811 22:19:14.890327 2548 leader_election.cc:288] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039), e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803)
W20260811 22:19:14.891281 2064 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e889c3b6dcad43088aee18d0d92399b8 (127.17.198.2:43039): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
W20260811 22:19:14.891498 2066 leader_election.cc:341] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e679a73a13244e5c881d28b77b69859a (127.17.198.3:35803): Not found: Tablet not found: eee68c3326a44972ad1a18b985b8ed5d
I20260811 22:19:14.891606 2066 leader_election.cc:302] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67dbc860a99d4115938224c8b9f0846f; no voters: e679a73a13244e5c881d28b77b69859a, e889c3b6dcad43088aee18d0d92399b8
I20260811 22:19:14.891785 2548 raft_consensus.cc:2726] T eee68c3326a44972ad1a18b985b8ed5d P 67dbc860a99d4115938224c8b9f0846f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:15.246966 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:15.246455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:53093)
0811 22:19:15.246763 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:15.246850 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":108,"threads_started":1}
I20260811 22:19:15.437130 2550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:15.436256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:37687)
0811 22:19:15.436645 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:15.436994 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":96,"threads_started":1}
I20260811 22:19:15.649456 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:15.648735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:43731)
0811 22:19:15.649227 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:15.649294 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":250,"threads_started":1}
I20260811 22:19:16.248235 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.247734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:42891)
0811 22:19:16.248054 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.248132 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
I20260811 22:19:16.438640 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.438178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:60097)
0811 22:19:16.438482 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.438540 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":108,"threads_started":1}
I20260811 22:19:16.651014 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.650460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.3:54695)
0811 22:19:16.650815 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.650908 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":114,"threads_started":1}
I20260811 22:19:17.249949 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.249532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.1:45113)
0811 22:19:17.249786 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:17.249841 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260811 22:19:17.440258 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.439622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:35391 (local address 127.17.198.2:40605)
0811 22:19:17.440028 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:17.440110 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:35391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":91,"threads_started":1}
I20260811 22:19:17.631060 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2006
I20260811 22:19:17.641036 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2195
I20260811 22:19:17.650617 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2350
2026-08-11T22:19:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19732 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:17 dist-test-slave-77v9 krb5kdc[2560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:17 dist-test-slave-77v9 krb5kdc[2560](info): set up 2 sockets
Aug 11 22:19:17 dist-test-slave-77v9 krb5kdc[2560](info): commencing operation
krb5kdc: starting...
W20260811 22:19:18.823246 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:19:18 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:19:18.969939 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:19.094868 2575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:19.095191 2575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:19.095235 2575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:19.099387 2575 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:19:19.099467 2575 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:19:19.099484 2575 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:19:19.099498 2575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:19.099517 2575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:19.099536 2575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:19.099552 2575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:19.099568 2575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:19.105583 2575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:19.106848 2575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:19.107960 2575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:19.117671 2585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:19.117663 2583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.117801 2575 server_base.cc:1034] running on GCE node
W20260811 22:19:19.118058 2582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.118677 2575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:19.119871 2575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:19.121085 2575 hybrid_clock.cc:648] HybridClock initialized: now 1786486759121057 us; error 43 us; skew 500 ppm
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:19.124344 2575 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:19:19.125475 2575 webserver.cc:460] Webserver started at http://127.17.198.62:42883/ using document root <none> and password file <none>
I20260811 22:19:19.126127 2575 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:19.126233 2575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:19.126504 2575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:19.128767 2575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "e055ec6f7cff425288fcc7b315c8a206"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "920c7ae3cc937b22aa66615b11afb37a"
server_key_iv: "c0835400d1058f6d58a4489d936c170f"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.129508 2575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "e055ec6f7cff425288fcc7b315c8a206"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "920c7ae3cc937b22aa66615b11afb37a"
server_key_iv: "c0835400d1058f6d58a4489d936c170f"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.134815 2575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:19:19.138259 2593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.139920 2575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:19.140143 2575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "e055ec6f7cff425288fcc7b315c8a206"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "920c7ae3cc937b22aa66615b11afb37a"
server_key_iv: "c0835400d1058f6d58a4489d936c170f"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.140303 2575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:19.172688 2575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:19.176635 2575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:19.176921 2575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:19.185402 2575 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37875
I20260811 22:19:19.185761 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37875 every 8 connection(s)
I20260811 22:19:19.186587 2575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:19.189003 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2575
I20260811 22:19:19.189139 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:19:19.189477 18200 external_mini_cluster.cc:1428] Setting key b82650c9e6b95108804c4b713b859950
I20260811 22:19:19.189991 2657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.197789 2657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap starting.
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:19.200874 2657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.201845 2657 log.cc:824] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:19.204552 2657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: No bootstrap required, opened a new log
I20260811 22:19:19.205838 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.191058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.17.198.62:37875)
0811 22:19:19.191508 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.191532 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.191568 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:19:19.192256 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.192274 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.192614 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.193011 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.193976 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.195042 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.196323 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.196924 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.197054 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.200225 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:19.200231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:19.200247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:19.200283 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:19.202388 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.203024 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.203031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.203037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.203184 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.203462 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.203465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.203467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.204013 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:19.204207 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.204476 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.204803 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":164,"threads_started":1}
I20260811 22:19:19.209080 2657 raft_consensus.cc:348] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:19.209321 2657 raft_consensus.cc:374] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.209355 2657 raft_consensus.cc:729] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.209722 2657 consensus_queue.cc:260] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:19.209836 2657 raft_consensus.cc:388] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:19.209875 2657 raft_consensus.cc:482] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:19.209947 2657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.210906 2657 raft_consensus.cc:504] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:19.211300 2657 leader_election.cc:302] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e055ec6f7cff425288fcc7b315c8a206; no voters:
I20260811 22:19:19.211679 2657 leader_election.cc:288] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:19.211830 2664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:19.212149 2664 raft_consensus.cc:686] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 LEADER]: Becoming Leader. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Running, Role: LEADER
I20260811 22:19:19.212508 2664 consensus_queue.cc:237] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:19.213259 2657 sys_catalog.cc:564] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:19.214447 2666 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e055ec6f7cff425288fcc7b315c8a206. Latest consensus state: current_term: 1 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:19:19.214612 2666 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:19.214452 2665 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:19:19.214974 2665 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:19.214998 2674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:19.218423 2674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:19.225281 2674 catalog_manager.cc:1347] Generated new cluster ID: 0c118ea6e3fc4a949d98454df640324d
I20260811 22:19:19.225431 2674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:19.250780 2674 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:19.252233 2674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:19.264312 2674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Generated new TSK 0
I20260811 22:19:19.265319 2674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:19:19.304113 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:19.429235 2689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:19.429632 2689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:19.429817 2689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:19.435380 2689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:19.435478 2689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:19.435586 2689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:19:19.442162 2689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:19.443384 2689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:19.444641 2689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:19.455533 2698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.455641 2689 server_base.cc:1034] running on GCE node
W20260811 22:19:19.455523 2696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:19.455525 2695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.456355 2689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:19.457151 2689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:19.458384 2689 hybrid_clock.cc:648] HybridClock initialized: now 1786486759458366 us; error 35 us; skew 500 ppm
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:19.461853 2689 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:19:19.462977 2689 webserver.cc:460] Webserver started at http://127.17.198.1:42703/ using document root <none> and password file <none>
I20260811 22:19:19.463634 2689 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:19.463742 2689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:19.464016 2689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:19.466259 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "37e6577157944daaa74f47dae2748554"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "7a5680869603b46fad0b037f34d75eef"
server_key_iv: "6a42285a43b4139a039ce490c3360e67"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.466930 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "37e6577157944daaa74f47dae2748554"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "7a5680869603b46fad0b037f34d75eef"
server_key_iv: "6a42285a43b4139a039ce490c3360e67"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.471534 2689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:19:19.474710 2708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.476370 2689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:19.476598 2689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "37e6577157944daaa74f47dae2748554"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "7a5680869603b46fad0b037f34d75eef"
server_key_iv: "6a42285a43b4139a039ce490c3360e67"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.476765 2689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:19.511040 2689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:19.515216 2689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:19.515519 2689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:19.516228 2689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:19.517233 2689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:19.517321 2689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.517413 2689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:19.517477 2689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.528355 2689 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:40779
I20260811 22:19:19.528370 2840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:40779 every 8 connection(s)
I20260811 22:19:19.529471 2689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:19.533260 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2689
I20260811 22:19:19.533433 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:19.533757 18200 external_mini_cluster.cc:1428] Setting key 507caaacbc299e45872129551efd74c5
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:19:19.546267 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.531555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:43155 (local address 127.17.198.62:37875)
0811 22:19:19.531722 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.531726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.532657 (+ 931us) server_negotiation.cc:407] Connection header received
0811 22:19:19.534338 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.534341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.534409 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.534531 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.535846 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.536391 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.537932 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.538214 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.538319 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.541912 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:19.541916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:19.541918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:19.541950 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:19.543606 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.544250 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.544253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.544255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.544360 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.544768 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.544771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.544773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.545151 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:19.545275 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.545902 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.546065 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:19:19.547235 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.531980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:43155)
0811 22:19:19.532488 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.532535 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.534092 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.534727 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.534734 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.535154 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:19.535686 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.535701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.536554 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.536558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.537698 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.537714 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.538407 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.538413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.538665 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.539431 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:19.539460 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:19.541662 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:19.543754 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.543761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.543779 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.544100 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.544471 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.544475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.544478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.544645 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.545274 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:19.545282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:19.545612 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.545829 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.546220 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:19:19.548861 2841 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:19.549171 2841 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:19.549845 2841 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:19.551705 2613 ts_manager.cc:194] Registered new tserver with Master: 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:19.552968 2613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:43155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:19:19.569360 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:19.694098 2846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:19.694429 2846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:19.694478 2846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:19.698987 2846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:19.699074 2846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:19.699138 2846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:19:19.705449 2846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:19.706700 2846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:19.707788 2846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:19.719205 2853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:19.719205 2855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.719458 2846 server_base.cc:1034] running on GCE node
W20260811 22:19:19.719200 2852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.720029 2846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:19.720774 2846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:19.722018 2846 hybrid_clock.cc:648] HybridClock initialized: now 1786486759722001 us; error 32 us; skew 500 ppm
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:19.725292 2846 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:19:19.726287 2846 webserver.cc:460] Webserver started at http://127.17.198.2:46129/ using document root <none> and password file <none>
I20260811 22:19:19.726934 2846 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:19.727047 2846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:19.727293 2846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:19.729615 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "5ffd936d39b9a28c93df2ea55ccf3d8b"
server_key_iv: "e41137f4e758fa43524ac4ef0032be28"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.730237 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "5ffd936d39b9a28c93df2ea55ccf3d8b"
server_key_iv: "e41137f4e758fa43524ac4ef0032be28"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.734282 2846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:19:19.737516 2863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.739166 2846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:19.739373 2846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "5ffd936d39b9a28c93df2ea55ccf3d8b"
server_key_iv: "e41137f4e758fa43524ac4ef0032be28"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.739537 2846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:19.777987 2846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:19.781996 2846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:19.782229 2846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:19.782928 2846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:19.783865 2846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:19.783943 2846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.784015 2846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:19.784066 2846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:19.794236 2846 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:32853
I20260811 22:19:19.794260 2989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:32853 every 8 connection(s)
I20260811 22:19:19.795369 2846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:19.798693 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2846
I20260811 22:19:19.798856 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:19.799197 18200 external_mini_cluster.cc:1428] Setting key 75d7b947139388a6b9f5048f76e517a1
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:19:19.814100 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.797825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:43315 (local address 127.17.198.62:37875)
0811 22:19:19.798056 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.798060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.799102 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 22:19:19.800768 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.800773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.800871 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.801013 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.802539 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.803419 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.805429 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.805832 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.805975 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.809349 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:19.809355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:19.809359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:19.809415 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:19.811335 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.812097 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.812104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.812109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.812212 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.812620 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.812623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.812624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.812928 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:19.813075 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.813723 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.813844 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:19:19.815426 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.798246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:43315)
0811 22:19:19.798891 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.798956 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.800458 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.801232 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.801241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.801803 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:19.802354 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.802370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.803627 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.803632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.805249 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.805261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.806028 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.806032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.806197 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.807006 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:19.807039 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:19.809107 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:19.811506 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.811512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.811528 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.811978 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.812337 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.812340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.812341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.812514 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.813106 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:19.813138 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:19.813470 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.813660 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.814247 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":178,"threads_started":1}
I20260811 22:19:19.816875 2990 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:19.817229 2990 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:19.817852 2990 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:19.819180 2613 ts_manager.cc:194] Registered new tserver with Master: 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853)
I20260811 22:19:19.819862 2613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:43315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:19:19.837360 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:19.960230 2995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:19.960564 2995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:19.960613 2995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:19.965157 2995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:19.965238 2995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:19.965301 2995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:19:19.970937 2995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:19.972147 2995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:19.973335 2995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:19.984851 3002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:19.984851 3001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.984997 2995 server_base.cc:1034] running on GCE node
W20260811 22:19:19.984851 3004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:19.985625 2995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:19.986310 2995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:19.987527 2995 hybrid_clock.cc:648] HybridClock initialized: now 1786486759987512 us; error 31 us; skew 500 ppm
Aug 11 22:19:19 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:19.990842 2995 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:19:19.991952 2995 webserver.cc:460] Webserver started at http://127.17.198.3:44335/ using document root <none> and password file <none>
I20260811 22:19:19.992611 2995 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:19.992717 2995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:19.993048 2995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:19.995441 2995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "39ef057a72ad431ab579cfde3d6d0257"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "053d8f8e33b391a7e544b58eab878007"
server_key_iv: "da52af9a302f4d41fd9131ca0975de39"
server_key_version: "encryptionkey@0"
I20260811 22:19:19.996129 2995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "39ef057a72ad431ab579cfde3d6d0257"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "053d8f8e33b391a7e544b58eab878007"
server_key_iv: "da52af9a302f4d41fd9131ca0975de39"
server_key_version: "encryptionkey@0"
I20260811 22:19:20.000586 2995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:19:20.003569 3011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:20.005127 2995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:20.005286 2995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "39ef057a72ad431ab579cfde3d6d0257"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "053d8f8e33b391a7e544b58eab878007"
server_key_iv: "da52af9a302f4d41fd9131ca0975de39"
server_key_version: "encryptionkey@0"
I20260811 22:19:20.005434 2995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:20.030988 2995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:20.034509 2995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:20.034762 2995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:20.035454 2995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:20.036362 2995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:20.036424 2995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:20.036497 2995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:20.036532 2995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:20.047221 2995 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:38705
I20260811 22:19:20.047312 3145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:38705 every 8 connection(s)
I20260811 22:19:20.048362 2995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:20.056984 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 2995
I20260811 22:19:20.057112 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:20.057431 18200 external_mini_cluster.cc:1428] Setting key 2f17a5a41999bb8dcf6e9fa481adaa2d
Aug 11 22:19:20 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:20.063310 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.050468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:41039 (local address 127.17.198.62:37875)
0811 22:19:20.050626 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.050630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.051485 (+ 855us) server_negotiation.cc:407] Connection header received
0811 22:19:20.052994 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.052998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.053073 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.053178 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:20.054458 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.055005 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.056213 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.056445 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.056574 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.059626 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:20.059630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:20.059635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:20.059695 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:20.061279 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.061806 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.061808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.061809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.061938 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.062253 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.062255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.062256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.062451 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:20.062538 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.063060 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.063182 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:19:20.064289 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.050838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:41039)
0811 22:19:20.051315 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.051362 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.052674 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.053367 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.053375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.053783 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:20.054287 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.054301 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.055152 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.055156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.056066 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.056077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.056641 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.056646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.056888 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.057593 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:20.057611 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:20.059447 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:20.061401 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:20.061407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:20.061422 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:20.061712 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:20.062025 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:20.062027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:20.062028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:20.062172 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:20.062546 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:20.062551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:20.062816 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.062993 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.063283 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260811 22:19:20.065693 3146 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:20.065927 3146 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:20.066411 3146 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:20.067641 2613 ts_manager.cc:194] Registered new tserver with Master: 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
I20260811 22:19:20.068315 2613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:41039
I20260811 22:19:20.071807 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:19:20.083452 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.074158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.17.198.62:37875)
0811 22:19:20.074299 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.074302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.074491 (+ 189us) server_negotiation.cc:407] Connection header received
0811 22:19:20.074577 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.074580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.074628 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.074705 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:20.075564 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.076094 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.077172 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.077441 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.077590 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.079445 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:20.079449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:20.079452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:20.079495 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:20.081467 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.082013 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.082019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.082021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.082161 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.082470 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.082473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.082474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.082758 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:20.082929 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.083186 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.083309 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:19:20.087047 2613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:20.090430 2613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:20.107164 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.101731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.17.198.3:38705)
0811 22:19:20.102130 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.102142 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.102164 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:19:20.102243 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.102248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.102445 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.102600 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:20.103051 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.103991 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.105695 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.106400 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.106550 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.106720 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.106899 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.106997 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":150,"threads_started":1}
I20260811 22:19:20.107149 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.100682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38705 (local address 127.0.0.1:50278)
0811 22:19:20.101517 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.101534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.101634 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.102765 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.102771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.102789 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:20.102903 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.102911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.104146 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.104149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.105557 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.105566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.106563 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.106567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.106675 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.106712 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.106856 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.106994 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":105,"thread_start_us":131,"threads_started":1}
I20260811 22:19:20.107875 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.100634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.17.198.2:32853)
0811 22:19:20.101017 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.101029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.101686 (+ 657us) server_negotiation.cc:407] Connection header received
0811 22:19:20.101775 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.101779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.102004 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.102175 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:20.102813 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.104008 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.105723 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.106572 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.106653 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.106792 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.107143 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.107703 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":135,"threads_started":1}
I20260811 22:19:20.108238 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.101251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.0.0.1:47708)
0811 22:19:20.102979 (+ 1728us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.102995 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.103086 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.103541 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.103544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.103563 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:20.103650 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.103656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.105273 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.105277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.106494 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.106506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.107810 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.107813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.107910 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.107939 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.108028 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.108107 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":99,"threads_started":1}
I20260811 22:19:20.108266 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.100465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:32853 (local address 127.0.0.1:55856)
0811 22:19:20.101286 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.101376 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.101530 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.102471 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.102478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.102507 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:20.102664 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.102685 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.104173 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.104176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.105574 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.105584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.106894 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.106897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.106988 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.107010 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.108100 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.108162 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":64,"thread_start_us":98,"threads_started":1}
I20260811 22:19:20.108657 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.101455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.17.198.1:40779)
0811 22:19:20.102601 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.102607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.103079 (+ 472us) server_negotiation.cc:407] Connection header received
0811 22:19:20.103202 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.103208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.103384 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.103525 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:20.103778 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.104922 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.106653 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.107508 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.107612 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.107745 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.108369 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.108495 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":72,"threads_started":1}
I20260811 22:19:20.110518 3063 tablet_service.cc:1467] Processing CreateTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 (DEFAULT_TABLE table=test-table [id=de6078222fb54e20beaccb466c9231cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.111616 3063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46b92fd9393438e9ba3e8eab75587b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.111246 2762 tablet_service.cc:1467] Processing CreateTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 (DEFAULT_TABLE table=test-table [id=de6078222fb54e20beaccb466c9231cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.111248 2916 tablet_service.cc:1467] Processing CreateTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 (DEFAULT_TABLE table=test-table [id=de6078222fb54e20beaccb466c9231cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.112243 2762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46b92fd9393438e9ba3e8eab75587b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.112243 2916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46b92fd9393438e9ba3e8eab75587b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.119321 3165 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Bootstrap starting.
I20260811 22:19:20.120050 3166 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Bootstrap starting.
I20260811 22:19:20.121167 3168 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Bootstrap starting.
I20260811 22:19:20.121690 3165 tablet_bootstrap.cc:654] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.122112 3166 tablet_bootstrap.cc:654] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.122900 3165 log.cc:824] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:20.122946 3166 log.cc:824] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:20.123756 3168 tablet_bootstrap.cc:654] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.124604 3168 log.cc:824] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:20.125262 3166 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: No bootstrap required, opened a new log
I20260811 22:19:20.125262 3165 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: No bootstrap required, opened a new log
I20260811 22:19:20.125473 3165 ts_tablet_manager.cc:1397] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:19:20.125473 3166 ts_tablet_manager.cc:1397] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 22:19:20.126499 3168 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: No bootstrap required, opened a new log
I20260811 22:19:20.126721 3168 ts_tablet_manager.cc:1397] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:19:20.129715 3168 raft_consensus.cc:348] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.129846 3165 raft_consensus.cc:348] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.130103 3168 raft_consensus.cc:374] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.130162 3168 raft_consensus.cc:729] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c909e93b3b41ee8b1b42fd3cbc3b10, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.130155 3165 raft_consensus.cc:374] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.130219 3165 raft_consensus.cc:729] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ef057a72ad431ab579cfde3d6d0257, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.130172 3166 raft_consensus.cc:348] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.130440 3166 raft_consensus.cc:374] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.130503 3166 raft_consensus.cc:729] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e6577157944daaa74f47dae2748554, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.130707 3168 consensus_queue.cc:260] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.130764 3165 consensus_queue.cc:260] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.130862 3166 consensus_queue.cc:260] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.131915 3168 ts_tablet_manager.cc:1428] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:19:20.132055 3166 ts_tablet_manager.cc:1428] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:19:20.132055 3165 ts_tablet_manager.cc:1428] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:19:20.132299 2990 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:20.132400 2841 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:20.132400 3146 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:20.133966 3172 raft_consensus.cc:482] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:20.134127 3172 raft_consensus.cc:504] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.135859 3172 leader_election.cc:288] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:20.141206 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.135648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:32853 (local address 127.17.198.3:59419)
0811 22:19:20.135849 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.135866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.135962 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.137294 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.137299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.137318 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:20.137427 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.137434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.138390 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.138394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.139794 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.139800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.140647 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.140651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.140783 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.140848 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.140939 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.141038 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":57}
I20260811 22:19:20.141279 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.136793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:59419 (local address 127.17.198.2:32853)
0811 22:19:20.137052 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.137057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.137070 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:19:20.137117 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.137122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.137176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.137261 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:20.137568 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.138269 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.139940 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.140508 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.140641 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.140694 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.140970 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.141057 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260811 22:19:20.141418 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.136669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:53405 (local address 127.17.198.1:40779)
0811 22:19:20.136792 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.136796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.136845 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:19:20.136886 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.136889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.136934 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.137010 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:20.137372 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.138179 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.139951 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.140653 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.140783 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.140852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.141212 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.141295 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:19:20.141566 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.136163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.17.198.3:53405)
0811 22:19:20.136506 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.136520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.136611 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.137075 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.137078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.137098 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:20.137211 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.137216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.138327 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.138333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.139782 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.139796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.140918 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.140923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.141029 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.141049 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.141120 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.141179 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":194,"thread_start_us":139,"threads_started":1}
I20260811 22:19:20.141947 2938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46b92fd9393438e9ba3e8eab75587b1" candidate_uuid: "39ef057a72ad431ab579cfde3d6d0257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" is_pre_election: true
I20260811 22:19:20.142184 2787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46b92fd9393438e9ba3e8eab75587b1" candidate_uuid: "39ef057a72ad431ab579cfde3d6d0257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e6577157944daaa74f47dae2748554" is_pre_election: true
I20260811 22:19:20.142344 2938 raft_consensus.cc:2454] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ef057a72ad431ab579cfde3d6d0257 in term 0.
I20260811 22:19:20.142446 2787 raft_consensus.cc:2454] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ef057a72ad431ab579cfde3d6d0257 in term 0.
I20260811 22:19:20.142930 3012 leader_election.cc:302] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 37e6577157944daaa74f47dae2748554, 39ef057a72ad431ab579cfde3d6d0257; no voters:
I20260811 22:19:20.143327 3175 raft_consensus.cc:2781] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:20.143425 3175 raft_consensus.cc:482] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:20.143517 3175 raft_consensus.cc:3037] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.144797 3175 raft_consensus.cc:504] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.145305 3175 leader_election.cc:288] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [CANDIDATE]: Term 1 election: Requested vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:20.145731 2787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46b92fd9393438e9ba3e8eab75587b1" candidate_uuid: "39ef057a72ad431ab579cfde3d6d0257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e6577157944daaa74f47dae2748554"
I20260811 22:19:20.145751 2938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46b92fd9393438e9ba3e8eab75587b1" candidate_uuid: "39ef057a72ad431ab579cfde3d6d0257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
I20260811 22:19:20.145888 2787 raft_consensus.cc:3037] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.145913 2938 raft_consensus.cc:3037] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.147068 2787 raft_consensus.cc:2454] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ef057a72ad431ab579cfde3d6d0257 in term 1.
I20260811 22:19:20.147076 2938 raft_consensus.cc:2454] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ef057a72ad431ab579cfde3d6d0257 in term 1.
I20260811 22:19:20.147473 3014 leader_election.cc:302] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257; no voters:
I20260811 22:19:20.147699 3175 raft_consensus.cc:2781] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:20.147825 3175 raft_consensus.cc:686] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 1 LEADER]: Becoming Leader. State: Replica: 39ef057a72ad431ab579cfde3d6d0257, State: Running, Role: LEADER
I20260811 22:19:20.148170 3175 consensus_queue.cc:237] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:20.152087 2612 catalog_manager.cc:5495] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "39ef057a72ad431ab579cfde3d6d0257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:20.174821 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.171010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.17.198.3:38705)
0811 22:19:20.171163 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.171167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.171182 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:19:20.171308 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.171310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.171366 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.171489 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:20.171725 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.172274 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.173250 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.173550 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.173639 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.173928 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:20.174420 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:20.174516 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.174622 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.174678 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:19:20.181412 2938 raft_consensus.cc:1261] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Refusing update from remote peer 39ef057a72ad431ab579cfde3d6d0257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.181412 2787 raft_consensus.cc:1261] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Refusing update from remote peer 39ef057a72ad431ab579cfde3d6d0257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.182261 3172 consensus_queue.cc:1035] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.182502 3175 consensus_queue.cc:1035] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.193367 3181 mvcc.cc:205] Tried to move back new op lower bound from 7317449769694564352 to 7317449769570201600. Current Snapshot: MvccSnapshot[applied={T|T < 7317449769694564352}]
I20260811 22:19:20.209582 2612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.209760 2612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.212445 2612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Sending DeleteTablet for 3 replicas of tablet f46b92fd9393438e9ba3e8eab75587b1
I20260811 22:19:20.213402 2916 tablet_service.cc:1514] Processing DeleteTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:55856
I20260811 22:19:20.213447 3063 tablet_service.cc:1514] Processing DeleteTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50278
I20260811 22:19:20.214067 3191 tablet_replica.cc:330] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: stopping tablet replica
I20260811 22:19:20.214041 2762 tablet_service.cc:1514] Processing DeleteTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:47708
I20260811 22:19:20.214442 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2689
I20260811 22:19:20.216733 3191 raft_consensus.cc:2229] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:20.217244 3191 raft_consensus.cc:2258] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.217969 3192 tablet_replica.cc:330] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: stopping tablet replica
I20260811 22:19:20.218394 3192 raft_consensus.cc:2229] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:20.218943 3192 raft_consensus.cc:2258] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.220211 3191 ts_tablet_manager.cc:1905] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.221930 3192 ts_tablet_manager.cc:1905] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.223272 3191 ts_tablet_manager.cc:1918] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:20.223389 3191 log.cc:1196] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/f46b92fd9393438e9ba3e8eab75587b1
I20260811 22:19:20.223747 3191 ts_tablet_manager.cc:1939] T f46b92fd9393438e9ba3e8eab75587b1 P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting consensus metadata
W20260811 22:19:20.224574 2594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:40779 (error 108)
I20260811 22:19:20.225333 3192 ts_tablet_manager.cc:1918] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:20.225543 3192 log.cc:1196] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/f46b92fd9393438e9ba3e8eab75587b1
I20260811 22:19:20.225672 2597 catalog_manager.cc:4841] TS 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): tablet f46b92fd9393438e9ba3e8eab75587b1 (table test-table [id=de6078222fb54e20beaccb466c9231cb]) successfully deleted
I20260811 22:19:20.225792 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2846
I20260811 22:19:20.228294 3192 ts_tablet_manager.cc:1939] T f46b92fd9393438e9ba3e8eab75587b1 P 39ef057a72ad431ab579cfde3d6d0257: Deleting consensus metadata
I20260811 22:19:20.230208 2594 catalog_manager.cc:4841] TS 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): tablet f46b92fd9393438e9ba3e8eab75587b1 (table test-table [id=de6078222fb54e20beaccb466c9231cb]) successfully deleted
I20260811 22:19:20.232117 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.231704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.0.0.1:47722)
0811 22:19:20.231896 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.231997 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 22:19:20.232342 2594 catalog_manager.cc:4584] TS 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f46b92fd9393438e9ba3e8eab75587b1: Network error: Client connection negotiation failed: client connection to 127.17.198.1:40779: connect: Connection refused (error 111)
I20260811 22:19:20.236145 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2995
I20260811 22:19:20.244361 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 2575
I20260811 22:19:40.254526 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--webserver_interface=127.17.198.62
--webserver_port=42883
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:40.375613 3196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:40.376004 3196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:40.376104 3196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:40.380461 3196 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:19:40.380600 3196 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:19:40.380635 3196 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:19:40.380678 3196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:40.380721 3196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:40.380796 3196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:40.380863 3196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:40.380906 3196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:40.386636 3196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=42883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:40.387866 3196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:40.389001 3196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:19:40.404124 3196 server_base.cc:1034] running on GCE node
W20260811 22:19:40.404062 3203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.404026 3206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.404045 3204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:40.405140 3196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:40.406363 3196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:40.407573 3196 hybrid_clock.cc:648] HybridClock initialized: now 1786486780407547 us; error 48 us; skew 500 ppm
Aug 11 22:19:40 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:40.411123 3196 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:19:40.412160 3196 webserver.cc:460] Webserver started at http://127.17.198.62:42883/ using document root <none> and password file <none>
I20260811 22:19:40.412899 3196 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:40.412995 3196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:40.417467 3196 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:40.419827 3213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:40.421334 3196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:40.421471 3196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "e055ec6f7cff425288fcc7b315c8a206"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "920c7ae3cc937b22aa66615b11afb37a"
server_key_iv: "c0835400d1058f6d58a4489d936c170f"
server_key_version: "encryptionkey@0"
I20260811 22:19:40.422116 3196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:40.441371 3196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:40.445474 3196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:40.445668 3196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:40.453846 3196 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37875
I20260811 22:19:40.454217 3275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37875 every 8 connection(s)
I20260811 22:19:40.455233 3196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:40.460254 3276 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:40.462117 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap starting.
I20260811 22:19:40.463490 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3196
I20260811 22:19:40.463967 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:40779
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=42703
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:40.486624 3276 log.cc:824] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:40.498365 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:40.498857 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap complete.
I20260811 22:19:40.504738 3276 raft_consensus.cc:348] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:40.505328 3276 raft_consensus.cc:729] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Initialized, Role: FOLLOWER
I20260811 22:19:40.505918 3276 consensus_queue.cc:260] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:40.506069 3276 raft_consensus.cc:388] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:40.506139 3276 raft_consensus.cc:482] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:40.506238 3276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:19:40.509047 3276 raft_consensus.cc:504] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:40.509495 3276 leader_election.cc:302] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e055ec6f7cff425288fcc7b315c8a206; no voters:
I20260811 22:19:40.509907 3276 leader_election.cc:288] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:19:40.510142 3281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:19:40.510444 3281 raft_consensus.cc:686] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 LEADER]: Becoming Leader. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Running, Role: LEADER
I20260811 22:19:40.510842 3281 consensus_queue.cc:237] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:19:40.511317 3276 sys_catalog.cc:564] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:40.512719 3283 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e055ec6f7cff425288fcc7b315c8a206. Latest consensus state: current_term: 2 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:19:40.512894 3283 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:40.513473 3282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:19:40.513577 3282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:40.517001 3299 catalog_manager.cc:1259] Loaded cluster ID: 0c118ea6e3fc4a949d98454df640324d
I20260811 22:19:40.517078 3299 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: loading cluster ID for follower catalog manager: success
I20260811 22:19:40.519634 3299 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: acquiring CA information for follower catalog manager: success
I20260811 22:19:40.521291 3299 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:19:40.521502 3292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:40.524263 3292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:40.524745 3292 catalog_manager.cc:1259] Loaded cluster ID: 0c118ea6e3fc4a949d98454df640324d
I20260811 22:19:40.524835 3292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:40.525563 3292 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:40.526081 3292 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Loaded TSK: 0
I20260811 22:19:40.541821 3292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Generated new TSK 1
I20260811 22:19:40.542179 3292 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:19:40.606328 3279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:40.606637 3279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:40.606685 3279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:40.610970 3279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:40.611044 3279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:40.611132 3279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:19:40.617280 3279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:40779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=42703
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:40.618398 3279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:40.619622 3279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:40.630028 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.630004 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.630004 3311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:40.630635 3279 server_base.cc:1034] running on GCE node
I20260811 22:19:40.631140 3279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:40.631985 3279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:40.633198 3279 hybrid_clock.cc:648] HybridClock initialized: now 1786486780633167 us; error 57 us; skew 500 ppm
Aug 11 22:19:40 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:40.636857 3279 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:19:40.638015 3279 webserver.cc:460] Webserver started at http://127.17.198.1:42703/ using document root <none> and password file <none>
I20260811 22:19:40.638743 3279 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:40.638962 3279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:40.643908 3279 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:19:40.646570 3319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:40.648067 3279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:19:40.648197 3279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "37e6577157944daaa74f47dae2748554"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "7a5680869603b46fad0b037f34d75eef"
server_key_iv: "6a42285a43b4139a039ce490c3360e67"
server_key_version: "encryptionkey@0"
I20260811 22:19:40.649103 3279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:40.674010 3279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:40.677363 3279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:40.677592 3279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:40.678267 3279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:40.679584 3327 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:19:40.683982 3279 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:19:40.684082 3279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:19:40.684176 3279 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:19:40.686115 3279 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:19:40.686193 3279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 22:19:40.686519 3327 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Bootstrap starting.
I20260811 22:19:40.697868 3279 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:40779
I20260811 22:19:40.698009 3454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:40779 every 8 connection(s)
I20260811 22:19:40.698988 3279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:40.703671 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3279
I20260811 22:19:40.704312 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:32853
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=46129
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:40.710227 3327 log.cc:824] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:19:40 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:40.718104 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.701293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:44753 (local address 127.17.198.62:37875)
0811 22:19:40.701677 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:40.701718 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:40.702121 (+ 403us) server_negotiation.cc:407] Connection header received
0811 22:19:40.703284 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:40.703296 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:40.703602 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:40.703968 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:40.705392 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.706561 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.708753 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.709342 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.709553 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.712523 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:40.712528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:40.712542 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:40.712576 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:40.714631 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:40.715273 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:40.715278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:40.715282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:40.715420 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:40.715782 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:40.715787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:40.715789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:40.716168 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:40.716325 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:40.716906 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 22:19:40.717183 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":140,"threads_started":1}
I20260811 22:19:40.718128 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.701364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:44753)
0811 22:19:40.701948 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:40.701992 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:40.702847 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:40.704088 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:40.704096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:40.704620 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:40.705187 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.705204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.706787 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.706791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.708520 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.708538 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.709530 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.709534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.709734 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.710466 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:40.710492 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:40.712272 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:40.714787 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:40.714794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:40.714808 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:40.715125 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:40.715528 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:40.715531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:40.715534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:40.715682 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:40.716293 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:40.716300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:40.716559 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 22:19:40.716766 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:19:40.717174 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":195,"threads_started":1}
I20260811 22:19:40.720332 3455 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:40.720605 3455 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:40.721201 3455 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:40.723479 3327 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:40.723626 3234 ts_manager.cc:194] Registered new tserver with Master: 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:40.724017 3327 tablet_bootstrap.cc:492] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Bootstrap complete.
I20260811 22:19:40.725801 3327 ts_tablet_manager.cc:1397] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Time spent bootstrapping tablet: real 0.040s user 0.033s sys 0.003s
I20260811 22:19:40.727545 3234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:44753
I20260811 22:19:40.729418 3327 raft_consensus.cc:348] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:40.730158 3327 raft_consensus.cc:729] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e6577157944daaa74f47dae2748554, State: Initialized, Role: FOLLOWER
I20260811 22:19:40.730728 3327 consensus_queue.cc:260] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:40.731845 3327 ts_tablet_manager.cc:1428] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:19:40.732261 3455 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:40.735451 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.726827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.0.0.1:38790)
0811 22:19:40.728672 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:40.728723 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:40.728968 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:40.730362 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:40.730368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:40.730397 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:40.730538 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.730546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.731836 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.731841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.733406 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.733420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.734769 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.734777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.734931 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.734969 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:19:40.735112 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 22:19:40.735268 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"thread_start_us":138,"threads_started":1}
I20260811 22:19:40.735661 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.727716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.17.198.1:40779)
0811 22:19:40.728271 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:40.728283 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:40.729200 (+ 917us) server_negotiation.cc:407] Connection header received
0811 22:19:40.729285 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:40.729292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:40.730020 (+ 728us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:40.730196 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:40.730690 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.731693 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.733575 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.734442 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.734570 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.734718 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:40.735151 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:19:40.735260 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":112,"threads_started":1}
I20260811 22:19:40.737190 3372 tablet_service.cc:1514] Processing DeleteTablet for tablet f46b92fd9393438e9ba3e8eab75587b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:38790
I20260811 22:19:40.738133 3466 tablet_replica.cc:330] stopping tablet replica
I20260811 22:19:40.738509 3466 raft_consensus.cc:2229] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:40.738808 3466 raft_consensus.cc:2258] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:40.739929 3466 ts_tablet_manager.cc:1905] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:40.744546 3466 ts_tablet_manager.cc:1918] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:40.744692 3466 log.cc:1196] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal/wals/f46b92fd9393438e9ba3e8eab75587b1
I20260811 22:19:40.745182 3466 ts_tablet_manager.cc:1939] T f46b92fd9393438e9ba3e8eab75587b1 P 37e6577157944daaa74f47dae2748554: Deleting consensus metadata
I20260811 22:19:40.747244 3214 catalog_manager.cc:4841] TS 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779): tablet f46b92fd9393438e9ba3e8eab75587b1 (table test-table [id=de6078222fb54e20beaccb466c9231cb]) successfully deleted
W20260811 22:19:40.842183 3460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:40.842548 3460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:40.842636 3460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:40.847091 3460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:40.847218 3460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:40.847319 3460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:19:40.853276 3460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:32853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=46129
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:40.854804 3460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:40.856035 3460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:19:40.867094 3460 server_base.cc:1034] running on GCE node
W20260811 22:19:40.867089 3471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.867086 3474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:40.867131 3472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:40.867800 3460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:40.868578 3460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:40.869802 3460 hybrid_clock.cc:648] HybridClock initialized: now 1786486780869768 us; error 49 us; skew 500 ppm
Aug 11 22:19:40 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:40.873034 3460 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:19:40.874047 3460 webserver.cc:460] Webserver started at http://127.17.198.2:46129/ using document root <none> and password file <none>
I20260811 22:19:40.874692 3460 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:40.874809 3460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:40.879233 3460 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:40.881788 3481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:40.883242 3460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:40.883431 3460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "5ffd936d39b9a28c93df2ea55ccf3d8b"
server_key_iv: "e41137f4e758fa43524ac4ef0032be28"
server_key_version: "encryptionkey@0"
I20260811 22:19:40.884047 3460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:40.895099 3460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:40.898658 3460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:40.898914 3460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:40.899559 3460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:40.900449 3460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:40.900524 3460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:40.900599 3460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:40.900650 3460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:40.910656 3460 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:32853
I20260811 22:19:40.910754 3613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:32853 every 8 connection(s)
I20260811 22:19:40.911875 3460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:40.915357 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3460
I20260811 22:19:40.915947 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:38705
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=44335
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:19:40 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:40.932412 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.914223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:57121 (local address 127.17.198.62:37875)
0811 22:19:40.914412 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:40.914416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:40.916103 (+ 1687us) server_negotiation.cc:407] Connection header received
0811 22:19:40.918362 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:40.918369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:40.918454 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:40.918567 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:40.920066 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.921034 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.923201 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.923462 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.923553 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.927673 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:40.927680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:40.927687 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:40.927736 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:40.929849 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:40.930456 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:40.930460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:40.930463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:40.930572 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:40.930933 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:40.930937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:40.930938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:40.931263 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:40.931376 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:40.932053 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 22:19:40.932213 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:19:40.933357 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.914646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:57121)
0811 22:19:40.915882 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:40.915949 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:40.917989 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:40.918782 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:40.918790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:40.919253 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:40.919853 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.919869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.921243 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.921250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.923001 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.923019 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.923665 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.923673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.923941 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.924667 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:40.924695 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:40.927360 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:40.930010 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:40.930016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:40.930033 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:40.930338 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:40.930676 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:40.930678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:40.930680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:40.930836 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:40.931455 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:40.931464 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:40.931776 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 22:19:40.931987 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:19:40.932289 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":123,"threads_started":1}
I20260811 22:19:40.934696 3614 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:40.935036 3614 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:40.935604 3614 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:40.936797 3234 ts_manager.cc:194] Registered new tserver with Master: 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853)
I20260811 22:19:40.937484 3234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:57121
W20260811 22:19:41.040365 3618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:41.040726 3618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:41.040813 3618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:41.045557 3618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:41.045634 3618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:41.045723 3618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:19:41.051365 3618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=44335
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:19:41.052533 3618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:41.053750 3618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:19:41.064385 3618 server_base.cc:1034] running on GCE node
W20260811 22:19:41.064518 3628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:41.064352 3624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:41.064352 3625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:41.065230 3618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:41.065982 3618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:41.067195 3618 hybrid_clock.cc:648] HybridClock initialized: now 1786486781067158 us; error 53 us; skew 500 ppm
Aug 11 22:19:41 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:41.070407 3618 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:19:41.071458 3618 webserver.cc:460] Webserver started at http://127.17.198.3:44335/ using document root <none> and password file <none>
I20260811 22:19:41.072104 3618 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:41.072248 3618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:41.076537 3618 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:41.078943 3635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:41.080430 3618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:19:41.080592 3618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "39ef057a72ad431ab579cfde3d6d0257"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "053d8f8e33b391a7e544b58eab878007"
server_key_iv: "da52af9a302f4d41fd9131ca0975de39"
server_key_version: "encryptionkey@0"
I20260811 22:19:41.081302 3618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:41.106914 3618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:41.110756 3618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:41.110970 3618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:41.111646 3618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:41.112535 3618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:41.112583 3618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:41.112624 3618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:41.112684 3618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:41.122249 3618 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:38705
I20260811 22:19:41.122357 3776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:38705 every 8 connection(s)
I20260811 22:19:41.123353 3618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:41.125473 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3618
Aug 11 22:19:41 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:19:41.135191 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.126229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.17.198.62:37875)
0811 22:19:41.126458 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.126462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.126801 (+ 339us) server_negotiation.cc:407] Connection header received
0811 22:19:41.126960 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.126964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.127012 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.127096 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.128173 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.128733 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.129664 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.129929 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.130053 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.131514 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:41.131518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:41.131520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:41.131554 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:41.133339 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.133779 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.133785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.133790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.133875 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.134238 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.134242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.134244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.134435 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:41.134660 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.134836 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.134970 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260811 22:19:41.138728 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.125446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:43595 (local address 127.17.198.62:37875)
0811 22:19:41.125644 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.125649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.126677 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 22:19:41.128557 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.128560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.128609 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.128684 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.129901 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.130411 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.131580 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.131805 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.131917 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.134702 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:41.134712 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:41.134715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:41.134739 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:41.136445 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.137106 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.137109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.137112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.137213 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.137657 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.137660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.137662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.137869 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:41.137953 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.138488 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.138599 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":79}
I20260811 22:19:41.139618 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.125908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:43595)
0811 22:19:41.126505 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.126573 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.128338 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.128899 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.128906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.129255 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:41.129755 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.129768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.130551 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.130555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.131440 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.131450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.131964 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.131968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.132150 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.132651 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:41.132669 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:41.134510 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:41.136575 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:41.136582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:41.136599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:41.137002 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:41.137328 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:41.137330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:41.137331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:41.137487 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:41.137980 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:41.137985 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:41.138234 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 22:19:41.138393 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 22:19:41.138672 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":161,"threads_started":1}
I20260811 22:19:41.141012 3780 heartbeater.cc:347] Connected to a master server at 127.17.198.62:37875
I20260811 22:19:41.141263 3780 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:41.141773 3780 heartbeater.cc:511] Master 127.17.198.62:37875 requested a full tablet report, sending...
I20260811 22:19:41.142908 3234 ts_manager.cc:194] Registered new tserver with Master: 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
I20260811 22:19:41.143448 3234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:43595
I20260811 22:19:41.149848 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:19:41.154079 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.150495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.17.198.62:37875)
0811 22:19:41.150671 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.150678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.150828 (+ 150us) server_negotiation.cc:407] Connection header received
0811 22:19:41.150986 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.150988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.151096 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.151187 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:41.151430 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.151964 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.152946 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.153162 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.153247 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.153577 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:41.153725 (+ 148us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:19:41.153941 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56768: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:19:41.154174 3457 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56768: token expired
W20260811 22:19:41.154584 3153 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.198.62:37875)
I20260811 22:19:41.162729 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.154976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.17.198.62:37875)
0811 22:19:41.155106 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.155111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.155124 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:19:41.155209 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.155212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.155260 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.155347 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.156003 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.156494 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.157404 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.157619 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.157727 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.159142 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:41.159145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:41.159148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:41.159181 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:41.160633 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.161187 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.161192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.161194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.161294 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:41.161609 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:41.161613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:41.161615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:41.161899 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:41.162062 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.162401 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.162545 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:19:41.165855 3234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:41.169567 3234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:41.178396 3372 tablet_service.cc:1467] Processing CreateTablet for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac (DEFAULT_TABLE table=test-table [id=afc6067999e84acca34231c270900769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:41.179129 3372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:41.182001 3327 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Bootstrap starting.
I20260811 22:19:41.182756 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.177005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:32853 (local address 127.0.0.1:35278)
0811 22:19:41.177189 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.177205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.177399 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.178127 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.178130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.178148 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:41.178257 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.178263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.179869 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.179872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.181029 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.181042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.182303 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.182307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.182415 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.182446 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:19:41.182501 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 22:19:41.182586 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 22:19:41.183012 3327 tablet_bootstrap.cc:654] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:41.183799 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.177339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38705 (local address 127.0.0.1:51598)
0811 22:19:41.177568 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.177582 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.177675 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.179090 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.179094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.179113 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:41.179213 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.179220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.180491 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.180498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.181843 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.181855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.183349 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.183353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.183466 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.183495 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:19:41.183574 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:19:41.183647 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260811 22:19:41.183817 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.177062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35278 (local address 127.17.198.2:32853)
0811 22:19:41.177545 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.177561 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.177588 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:19:41.177680 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.177688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.177953 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.178156 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:41.178380 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.179721 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.181157 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.182150 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.183289 (+ 1139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.183452 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.183509 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.183628 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":165,"threads_started":1}
I20260811 22:19:41.184258 3327 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: No bootstrap required, opened a new log
I20260811 22:19:41.184336 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.177853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.17.198.3:38705)
0811 22:19:41.178400 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.178421 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.178446 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:19:41.178542 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.178547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.178737 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.178910 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:41.179350 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.180364 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.182021 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.182961 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.183089 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.183272 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.184030 (+ 758us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.184156 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":195,"threads_started":1}
I20260811 22:19:41.184361 3327 ts_tablet_manager.cc:1397] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:19:41.184785 3327 raft_consensus.cc:348] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.184933 3327 raft_consensus.cc:374] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:41.184983 3327 raft_consensus.cc:729] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e6577157944daaa74f47dae2748554, State: Initialized, Role: FOLLOWER
I20260811 22:19:41.185118 3327 consensus_queue.cc:260] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.185375 3327 ts_tablet_manager.cc:1428] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:19:41.187515 3538 tablet_service.cc:1467] Processing CreateTablet for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac (DEFAULT_TABLE table=test-table [id=afc6067999e84acca34231c270900769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:41.187530 3695 tablet_service.cc:1467] Processing CreateTablet for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac (DEFAULT_TABLE table=test-table [id=afc6067999e84acca34231c270900769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:41.188606 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:41.188695 3695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:41.195902 3794 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Bootstrap starting.
I20260811 22:19:41.196766 3796 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Bootstrap starting.
I20260811 22:19:41.199291 3794 tablet_bootstrap.cc:654] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:41.199570 3796 tablet_bootstrap.cc:654] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:41.200193 3794 log.cc:824] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:41.200402 3796 log.cc:824] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Log is configured to *not* fsync() on all Append() calls
W20260811 22:19:41.200698 3456 tablet.cc:2369] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:41.202720 3796 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: No bootstrap required, opened a new log
I20260811 22:19:41.202987 3796 ts_tablet_manager.cc:1397] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:19:41.203142 3794 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: No bootstrap required, opened a new log
I20260811 22:19:41.203382 3794 ts_tablet_manager.cc:1397] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:19:41.206194 3796 raft_consensus.cc:348] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.206462 3796 raft_consensus.cc:374] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:41.206542 3796 raft_consensus.cc:729] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ef057a72ad431ab579cfde3d6d0257, State: Initialized, Role: FOLLOWER
I20260811 22:19:41.206632 3794 raft_consensus.cc:348] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.206961 3794 raft_consensus.cc:374] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:41.207033 3794 raft_consensus.cc:729] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c909e93b3b41ee8b1b42fd3cbc3b10, State: Initialized, Role: FOLLOWER
I20260811 22:19:41.207064 3796 consensus_queue.cc:260] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.207485 3794 consensus_queue.cc:260] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.207855 3780 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:41.208210 3614 heartbeater.cc:503] Master 127.17.198.62:37875 was elected leader, sending a full tablet report...
I20260811 22:19:41.208607 3794 ts_tablet_manager.cc:1428] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 22:19:41.209226 3796 ts_tablet_manager.cc:1428] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 22:19:41.324302 3800 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:41.324545 3800 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.325826 3800 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705), 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:41.330982 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.326089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38705 (local address 127.17.198.2:37983)
0811 22:19:41.326300 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.326318 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.326453 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.326920 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.326923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.326949 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:41.327103 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.327114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.328123 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.328126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.329609 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.329622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.330481 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.330485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.330586 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.330609 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:19:41.330688 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:19:41.330814 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":34}
I20260811 22:19:41.330947 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.326319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:37983 (local address 127.17.198.3:38705)
0811 22:19:41.326494 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.326502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.326528 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:19:41.326595 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.326599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.326659 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.326751 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:41.327244 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.327975 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.329750 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.330313 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.330445 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.330501 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.330733 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.330814 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:19:41.331291 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.326394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.17.198.2:45491)
0811 22:19:41.326922 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.326936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.327059 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.327415 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.327418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.327438 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:41.327551 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.327557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.328460 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.328463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.329878 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.329887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.330817 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.330820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.330935 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.330960 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:19:41.331051 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:19:41.331129 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":182,"threads_started":1}
I20260811 22:19:41.331331 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.326558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:45491 (local address 127.17.198.1:40779)
0811 22:19:41.326834 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.326837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.327053 (+ 216us) server_negotiation.cc:407] Connection header received
0811 22:19:41.327160 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.327162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.327239 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.327343 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:41.327692 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.328327 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.330017 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.330640 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.330785 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.330831 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.331092 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.331182 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":70,"threads_started":1}
I20260811 22:19:41.331885 3718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789c3cfb9f0f4c0192681dd45a3ad4ac" candidate_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39ef057a72ad431ab579cfde3d6d0257" is_pre_election: true
I20260811 22:19:41.332028 3400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789c3cfb9f0f4c0192681dd45a3ad4ac" candidate_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e6577157944daaa74f47dae2748554" is_pre_election: true
I20260811 22:19:41.332270 3400 raft_consensus.cc:2454] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c909e93b3b41ee8b1b42fd3cbc3b10 in term 0.
I20260811 22:19:41.332257 3718 raft_consensus.cc:2454] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c909e93b3b41ee8b1b42fd3cbc3b10 in term 0.
I20260811 22:19:41.332867 3482 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 37e6577157944daaa74f47dae2748554; no voters:
I20260811 22:19:41.333242 3800 raft_consensus.cc:2781] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:41.333382 3800 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:41.333451 3800 raft_consensus.cc:3037] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:41.334826 3800 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.335204 3800 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [CANDIDATE]: Term 1 election: Requested vote from peers 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705), 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779)
I20260811 22:19:41.335688 3718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789c3cfb9f0f4c0192681dd45a3ad4ac" candidate_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39ef057a72ad431ab579cfde3d6d0257"
I20260811 22:19:41.335762 3400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789c3cfb9f0f4c0192681dd45a3ad4ac" candidate_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e6577157944daaa74f47dae2748554"
I20260811 22:19:41.335841 3718 raft_consensus.cc:3037] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:41.335954 3400 raft_consensus.cc:3037] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:41.337098 3718 raft_consensus.cc:2454] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c909e93b3b41ee8b1b42fd3cbc3b10 in term 1.
I20260811 22:19:41.337306 3400 raft_consensus.cc:2454] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c909e93b3b41ee8b1b42fd3cbc3b10 in term 1.
I20260811 22:19:41.337514 3482 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257; no voters:
I20260811 22:19:41.337713 3800 raft_consensus.cc:2781] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:41.337982 3800 raft_consensus.cc:686] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 LEADER]: Becoming Leader. State: Replica: 08c909e93b3b41ee8b1b42fd3cbc3b10, State: Running, Role: LEADER
I20260811 22:19:41.338372 3800 consensus_queue.cc:237] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:19:41.342180 3233 catalog_manager.cc:5495] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } health_report { overall_health: UNKNOWN } } }
W20260811 22:19:41.374387 3781 tablet.cc:2369] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:41.380411 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.376708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.17.198.2:32853)
0811 22:19:41.376918 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.376922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.376937 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:19:41.377042 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.377044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.377104 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.377195 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:41.377442 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.378003 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.378886 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.379097 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.379178 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.379479 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:41.379904 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:41.380008 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:41.380145 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:19:41.380200 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:19:41.387100 3718 raft_consensus.cc:1261] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Refusing update from remote peer 08c909e93b3b41ee8b1b42fd3cbc3b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:41.387100 3400 raft_consensus.cc:1261] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Refusing update from remote peer 08c909e93b3b41ee8b1b42fd3cbc3b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:41.387948 3800 consensus_queue.cc:1035] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:41.388221 3803 consensus_queue.cc:1035] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:41.394693 3808 mvcc.cc:205] Tried to move back new op lower bound from 7317449856552407040 to 7317449856365182976. Current Snapshot: MvccSnapshot[applied={T|T < 7317449856552407040}]
I20260811 22:19:41.405987 3232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:41.406213 3232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:41.409013 3232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Sending DeleteTablet for 3 replicas of tablet 789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:19:41.410015 3695 tablet_service.cc:1514] Processing DeleteTablet for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:41 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:51598
I20260811 22:19:41.409993 3538 tablet_service.cc:1514] Processing DeleteTablet for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:41 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:35278
I20260811 22:19:41.410594 3819 tablet_replica.cc:330] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: stopping tablet replica
I20260811 22:19:41.410678 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 3279
I20260811 22:19:41.411017 3819 raft_consensus.cc:2229] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:41.411422 3819 raft_consensus.cc:2258] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:41.413805 3820 tablet_replica.cc:330] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: stopping tablet replica
I20260811 22:19:41.414271 3820 raft_consensus.cc:2229] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:41.414929 3820 raft_consensus.cc:2258] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:41.415089 3819 ts_tablet_manager.cc:1905] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:41.417960 3820 ts_tablet_manager.cc:1905] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:41.418017 3819 ts_tablet_manager.cc:1918] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:41.418116 3819 log.cc:1196] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:19:41.418485 3819 ts_tablet_manager.cc:1939] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 39ef057a72ad431ab579cfde3d6d0257: Deleting consensus metadata
I20260811 22:19:41.419939 3214 catalog_manager.cc:4841] TS 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): tablet 789c3cfb9f0f4c0192681dd45a3ad4ac (table test-table [id=afc6067999e84acca34231c270900769]) successfully deleted
I20260811 22:19:41.421001 3820 ts_tablet_manager.cc:1918] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:41.421125 3820 log.cc:1196] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:19:41.421483 3820 ts_tablet_manager.cc:1939] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 08c909e93b3b41ee8b1b42fd3cbc3b10: Deleting consensus metadata
W20260811 22:19:41.421567 3214 connection.cc:535] client connection to 127.17.198.1:40779 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 22:19:41.421746 3214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:19:41.422855 3217 catalog_manager.cc:4841] TS 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): tablet 789c3cfb9f0f4c0192681dd45a3ad4ac (table test-table [id=afc6067999e84acca34231c270900769]) successfully deleted
W20260811 22:19:41.423060 3791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:41.422740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:40779 (local address 127.0.0.1:38792)
0811 22:19:41.422905 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.422977 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:40779: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:19:41.423336 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 3460
W20260811 22:19:41.423271 3214 catalog_manager.cc:4584] TS 37e6577157944daaa74f47dae2748554 (127.17.198.1:40779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 789c3cfb9f0f4c0192681dd45a3ad4ac: Network error: Client connection negotiation failed: client connection to 127.17.198.1:40779: connect: Connection reset by peer (error 104)
I20260811 22:19:41.438716 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 3618
I20260811 22:19:41.449410 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 3196
I20260811 22:20:01.462030 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--webserver_interface=127.17.198.62
--webserver_port=42883
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:01.578734 3822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:01.579056 3822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:01.579149 3822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:01.583369 3822 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:20:01.583441 3822 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:20:01.583458 3822 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:20:01.583473 3822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:01.583492 3822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:01.583510 3822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:01.583564 3822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:01.583600 3822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:01.589270 3822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=42883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:01.590497 3822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:01.591604 3822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:01.600610 3822 server_base.cc:1034] running on GCE node
W20260811 22:20:01.600593 3830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:01.600576 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:01.600636 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:01.601595 3822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:01.602816 3822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:01.604025 3822 hybrid_clock.cc:648] HybridClock initialized: now 1786486801604012 us; error 31 us; skew 500 ppm
Aug 11 22:20:01 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486801, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:01.607349 3822 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:01.608333 3822 webserver.cc:460] Webserver started at http://127.17.198.62:42883/ using document root <none> and password file <none>
I20260811 22:20:01.609037 3822 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:01.609144 3822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:01.613462 3822 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:20:01.615754 3837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:01.617141 3822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:01.617293 3822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "e055ec6f7cff425288fcc7b315c8a206"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "920c7ae3cc937b22aa66615b11afb37a"
server_key_iv: "c0835400d1058f6d58a4489d936c170f"
server_key_version: "encryptionkey@0"
I20260811 22:20:01.617887 3822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:01.634231 3822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:01.637504 3822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:01.637740 3822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:01.645769 3822 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37875
I20260811 22:20:01.646123 3899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37875 every 8 connection(s)
I20260811 22:20:01.646981 3822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:01.650285 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3822
I20260811 22:20:01.650764 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:40779
--local_ip_for_outbound_sockets=127.17.198.1
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=42703
--webserver_interface=127.17.198.1
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:20:01.651567 3900 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:20:01.653461 3900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap starting.
I20260811 22:20:01.671885 3900 log.cc:824] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:01.683631 3900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:20:01.684111 3900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Bootstrap complete.
I20260811 22:20:01.688484 3900 raft_consensus.cc:348] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:20:01.689021 3900 raft_consensus.cc:729] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Initialized, Role: FOLLOWER
I20260811 22:20:01.689579 3900 consensus_queue.cc:260] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:20:01.689735 3900 raft_consensus.cc:388] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:01.689828 3900 raft_consensus.cc:482] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:01.689934 3900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:20:01.691939 3900 raft_consensus.cc:504] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:20:01.692291 3900 leader_election.cc:302] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e055ec6f7cff425288fcc7b315c8a206; no voters:
I20260811 22:20:01.692632 3900 leader_election.cc:288] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:20:01.692801 3906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:20:01.693121 3906 raft_consensus.cc:686] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [term 3 LEADER]: Becoming Leader. State: Replica: e055ec6f7cff425288fcc7b315c8a206, State: Running, Role: LEADER
I20260811 22:20:01.693442 3906 consensus_queue.cc:237] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } }
I20260811 22:20:01.693974 3900 sys_catalog.cc:564] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:01.694471 3906 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e055ec6f7cff425288fcc7b315c8a206. Latest consensus state: current_term: 3 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:20:01.694584 3906 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:01.694873 3907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e055ec6f7cff425288fcc7b315c8a206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e055ec6f7cff425288fcc7b315c8a206" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37875 } } }
I20260811 22:20:01.694998 3907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:01.695571 3911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:01.700845 3911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:01.701625 3911 catalog_manager.cc:1259] Loaded cluster ID: 0c118ea6e3fc4a949d98454df640324d
I20260811 22:20:01.701680 3911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:01.703425 3911 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:01.704059 3911 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Loaded TSK: 0
I20260811 22:20:01.704126 3911 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e055ec6f7cff425288fcc7b315c8a206: Loaded TSK: 1
F20260811 22:20:01.704752 3911 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb8044bb9ad google::LogMessage::Fail() at ??:0
@ 0x7fb8044be88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb8044bb4de google::LogMessage::Flush() at ??:0
@ 0x7fb8044bca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb80717f18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb807177f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb80ee207bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb80edfa99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb80edfb178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fb80ee3cdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb80ee5b82b std::function<>::operator()() at ??:0
@ 0x7fb80edfab4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fb80edfbf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb80edf8579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fb80ee3c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb80670d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb80670dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fb80670f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb8066fe512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb807d426db start_thread at ??:0
@ 0x7fb802fb961f clone at ??:0
W20260811 22:20:01.813746 3904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:01.814051 3904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:01.814098 3904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:01.818593 3904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:01.818668 3904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:01.818769 3904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:01.824488 3904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:40779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=42703
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:01.825742 3904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:01.826869 3904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:01.837563 3957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:01.837555 3953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:01.837555 3955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:01.837705 3904 server_base.cc:1034] running on GCE node
I20260811 22:20:01.838582 3904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:01.839341 3904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:01.840569 3904 hybrid_clock.cc:648] HybridClock initialized: now 1786486801840542 us; error 50 us; skew 500 ppm
Aug 11 22:20:01 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486801, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:01.843686 3904 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:20:01.844755 3904 webserver.cc:460] Webserver started at http://127.17.198.1:42703/ using document root <none> and password file <none>
I20260811 22:20:01.845408 3904 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:01.845477 3904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:01.849924 3904 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:20:01.852546 3965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:01.854236 3904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:01.854444 3904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "37e6577157944daaa74f47dae2748554"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "7a5680869603b46fad0b037f34d75eef"
server_key_iv: "6a42285a43b4139a039ce490c3360e67"
server_key_version: "encryptionkey@0"
I20260811 22:20:01.855134 3904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:01.868762 3904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:01.872745 3904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:01.873028 3904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:01.873754 3904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:01.875026 3974 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:20:01.881170 3904 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:20:01.881291 3904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260811 22:20:01.881378 3904 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:20:01.883279 3904 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:20:01.883370 3904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:20:01.883721 3974 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Bootstrap starting.
I20260811 22:20:01.895560 3904 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:40779
I20260811 22:20:01.895891 4101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:40779 every 8 connection(s)
I20260811 22:20:01.896723 3904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:01.900197 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 3904
I20260811 22:20:01.900730 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:32853
--local_ip_for_outbound_sockets=127.17.198.2
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=46129
--webserver_interface=127.17.198.2
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:20:01.908322 3974 log.cc:824] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:01.918654 3974 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:20:01.919209 3974 tablet_bootstrap.cc:492] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Bootstrap complete.
I20260811 22:20:01.920856 3974 ts_tablet_manager.cc:1397] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Time spent bootstrapping tablet: real 0.037s user 0.023s sys 0.012s
I20260811 22:20:01.924707 3974 raft_consensus.cc:348] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:01.925400 3974 raft_consensus.cc:729] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e6577157944daaa74f47dae2748554, State: Initialized, Role: FOLLOWER
I20260811 22:20:01.925994 3974 consensus_queue.cc:260] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:01.927197 3974 ts_tablet_manager.cc:1428] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 22:20:01.957893 4112 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:01.899280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:39903)
0811 22:20:01.899936 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:01.900009 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:01.901450 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:01.956886 (+ 55436us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":166,"threads_started":1}
W20260811 22:20:01.958575 4103 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:37875 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:37875: Client connection negotiation failed: client connection to 127.17.198.62:37875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:20:01.959438 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:01.959113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:55839)
0811 22:20:01.959272 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:01.959365 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:20:01.960223 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:01.959995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:44197)
0811 22:20:01.960113 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:01.960166 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:20:01.960479 4103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:20:02.025727 4113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:02.026046 4113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:02.026125 4113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:02.030740 4113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:02.030820 4113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:02.030892 4113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:02.036989 4113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:32853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=46129
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:02.038249 4113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:02.039368 4113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:02.050205 4122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:02.050266 4121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:02.050205 4125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:02.050702 4113 server_base.cc:1034] running on GCE node
I20260811 22:20:02.051232 4113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:02.051960 4113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:02.053182 4113 hybrid_clock.cc:648] HybridClock initialized: now 1786486802053164 us; error 29 us; skew 500 ppm
Aug 11 22:20:02 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486802, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:02.056419 4113 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:20:02.057585 4113 webserver.cc:460] Webserver started at http://127.17.198.2:46129/ using document root <none> and password file <none>
I20260811 22:20:02.058243 4113 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:02.058349 4113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:02.063179 4113 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:20:02.065740 4132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.067214 4113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:02.067404 4113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "5ffd936d39b9a28c93df2ea55ccf3d8b"
server_key_iv: "e41137f4e758fa43524ac4ef0032be28"
server_key_version: "encryptionkey@0"
I20260811 22:20:02.068150 4113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:02.083900 4113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:02.087493 4113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:02.087745 4113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:02.088443 4113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:02.089471 4113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:02.089558 4113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.089655 4113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:02.089715 4113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.100150 4113 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:32853
I20260811 22:20:02.100169 4255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:32853 every 8 connection(s)
I20260811 22:20:02.101420 4113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:02.105753 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.103816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:34631)
0811 22:20:02.104344 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.104472 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":139,"threads_started":1}
W20260811 22:20:02.106389 4256 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:37875 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:37875: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
I20260811 22:20:02.107117 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.106888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:52429)
0811 22:20:02.107005 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.107056 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:20:02.107767 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.107564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:46179)
0811 22:20:02.107675 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.107718 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:20:02.107959 4256 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:20:02.109589 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4113
I20260811 22:20:02.110055 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:38705
--local_ip_for_outbound_sockets=127.17.198.3
--tserver_master_addrs=127.17.198.62:37875
--webserver_port=44335
--webserver_interface=127.17.198.3
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:02.235590 4259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:02.235934 4259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:02.235986 4259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:02.240641 4259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:02.240736 4259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:02.240830 4259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:02.246523 4259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=44335
--tserver_master_addrs=127.17.198.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:02.247763 4259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:02.248833 4259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:02.259054 4265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:02.259054 4264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:02.259176 4268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:02.259455 4259 server_base.cc:1034] running on GCE node
I20260811 22:20:02.260102 4259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:02.260979 4259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:02.262218 4259 hybrid_clock.cc:648] HybridClock initialized: now 1786486802262187 us; error 35 us; skew 500 ppm
Aug 11 22:20:02 dist-test-slave-77v9 krb5kdc[2560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486802, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:02.265486 4259 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:20:02.266598 4259 webserver.cc:460] Webserver started at http://127.17.198.3:44335/ using document root <none> and password file <none>
I20260811 22:20:02.267218 4259 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:02.267323 4259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:02.271526 4259 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:20:02.274156 4277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.275555 4259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:20:02.275720 4259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "39ef057a72ad431ab579cfde3d6d0257"
format_stamp: "Formatted at 2026-08-11 22:19:19 on dist-test-slave-77v9"
server_key: "053d8f8e33b391a7e544b58eab878007"
server_key_iv: "da52af9a302f4d41fd9131ca0975de39"
server_key_version: "encryptionkey@0"
I20260811 22:20:02.276401 4259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:02.288760 4259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:02.291978 4259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:02.292220 4259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:02.292980 4259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:02.293883 4259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:02.293962 4259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.294025 4259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:02.294075 4259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:02.303684 4259 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:38705
I20260811 22:20:02.303712 4404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:38705 every 8 connection(s)
I20260811 22:20:02.304769 4259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:02.308595 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4259
I20260811 22:20:02.309182 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.307160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:37975)
0811 22:20:02.307671 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.307786 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":122,"threads_started":1}
W20260811 22:20:02.309809 4405 heartbeater.cc:644] Failed to heartbeat to 127.17.198.62:37875 (0 consecutive failures): Network error: Failed to ping master at 127.17.198.62:37875: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
I20260811 22:20:02.310472 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.310240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:40823)
0811 22:20:02.310367 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.310417 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:20:02.311146 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.310952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:37347)
0811 22:20:02.311052 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.311098 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:20:02.311379 4405 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:20:02.961750 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:02.961236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:56831)
0811 22:20:02.961565 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:02.961635 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260811 22:20:03.109444 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.108818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:56333)
0811 22:20:03.109208 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:03.109270 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260811 22:20:03.312549 4411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.312053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:54887)
0811 22:20:03.312321 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:03.312425 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260811 22:20:03.593185 4412 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:03.593472 4412 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:03.595597 4412 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
Aug 11 22:20:03 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486801, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.3@KRBTEST.COM
Aug 11 22:20:03 dist-test-slave-77v9 krb5kdc[2560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486801, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.2@KRBTEST.COM
I20260811 22:20:03.615687 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.595903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38705 (local address 127.17.198.1:38167)
0811 22:20:03.596317 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:03.596335 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:03.596497 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:03.598408 (+ 1911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:03.598414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:03.599048 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:03.599568 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:03.599586 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.601045 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.601050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:03.602112 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:03.602123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.602913 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:03.603186 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:03.607725 (+ 4539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:03.607752 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:03.610063 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:03.612849 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:03.612856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:03.612872 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:03.613269 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:03.613675 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:03.613678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:03.613680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:03.613837 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:03.614549 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:03.614556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:03.614869 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:20:03.615085 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:20:03.615497 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":184,"spinlock_wait_cycles":52480,"thread_start_us":129,"threads_started":1}
I20260811 22:20:03.615706 4416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.596144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:38167 (local address 127.17.198.3:38705)
0811 22:20:03.597035 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:03.597055 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:03.597088 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:20:03.597750 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:03.597756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:03.598061 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:03.598379 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:03.599768 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.600796 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602261 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.602764 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602953 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:03.610394 (+ 7441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:03.610401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:03.610417 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:03.610459 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:03.612646 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:03.613407 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:03.613421 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:03.613426 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:03.613562 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:03.613949 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:03.613952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:03.613954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:03.614415 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:03.614566 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:03.615202 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 22:20:03.615486 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":72,"threads_started":1}
I20260811 22:20:03.615770 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.596117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:39449 (local address 127.17.198.2:32853)
0811 22:20:03.596590 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:03.596622 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:03.596680 (+ 58us) server_negotiation.cc:407] Connection header received
0811 22:20:03.597535 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:03.597541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:03.597957 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:03.598247 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:03.599774 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.600967 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602260 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.602775 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602985 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:03.610618 (+ 7633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:03.610625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:03.610637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:03.610674 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:03.612682 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:03.613432 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:03.613439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:03.613444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:03.613589 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:03.614018 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:03.614024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:03.614026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:03.614454 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:03.614612 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:03.615207 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:20:03.615535 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":64,"threads_started":1}
I20260811 22:20:03.615847 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.595907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:32853 (local address 127.17.198.1:39449)
0811 22:20:03.596511 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:03.596523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:03.596607 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:03.598407 (+ 1800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:03.598414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:03.599047 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:03.599567 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:03.599585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.601131 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.601134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:03.602112 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:03.602123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:03.602946 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:03.602950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:03.603180 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:03.607725 (+ 4545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:03.607754 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:03.610381 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:03.612888 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:03.612891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:03.612894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:03.613278 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:03.613700 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:03.613704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:03.613707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:03.613846 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:03.614577 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:03.614581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:03.614869 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 22:20:03.615085 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:20:03.615601 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":437,"spinlock_wait_cycles":29184,"thread_start_us":86,"threads_started":1}
W20260811 22:20:03.619026 3966 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
W20260811 22:20:03.619221 3969 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:20:03.619356 3969 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37e6577157944daaa74f47dae2748554; no voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257
I20260811 22:20:03.619688 4412 raft_consensus.cc:2726] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:20:03.963042 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:03.962704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:58789)
0811 22:20:03.962860 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:03.962938 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:20:04.110983 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:04.110448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:50861)
0811 22:20:04.110781 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:04.110854 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":83,"threads_started":1}
I20260811 22:20:04.314213 4425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:04.313725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:37111)
0811 22:20:04.314031 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:04.314104 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":96,"threads_started":1}
I20260811 22:20:04.964634 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:04.964095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:43627)
0811 22:20:04.964402 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:04.964511 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":108,"threads_started":1}
I20260811 22:20:05.112515 4427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:05.111987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:49729)
0811 22:20:05.112304 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:05.112379 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":89,"threads_started":1}
I20260811 22:20:05.315909 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:05.315426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:40819)
0811 22:20:05.315737 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:05.315807 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":92,"threads_started":1}
I20260811 22:20:05.720352 4430 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:05.720520 4430 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:05.721016 4430 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
W20260811 22:20:05.722043 3966 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
W20260811 22:20:05.722195 3969 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:20:05.722271 3969 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37e6577157944daaa74f47dae2748554; no voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257
I20260811 22:20:05.722435 4430 raft_consensus.cc:2726] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:20:05.966112 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:05.965585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:35355)
0811 22:20:05.965939 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:05.966010 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":87,"threads_started":1}
I20260811 22:20:06.114038 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:06.113478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:34153)
0811 22:20:06.113832 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:06.113913 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260811 22:20:06.317610 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:06.317056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:55875)
0811 22:20:06.317424 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:06.317505 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":91,"threads_started":1}
I20260811 22:20:06.967880 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:06.967239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:53425)
0811 22:20:06.967631 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:06.967720 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":86,"threads_started":1}
I20260811 22:20:07.115432 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:07.114991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:48107)
0811 22:20:07.115259 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:07.115321 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260811 22:20:07.319161 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:07.318620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:59463)
0811 22:20:07.318975 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:07.319059 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":82,"threads_started":1}
I20260811 22:20:07.969718 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:07.969020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:41641)
0811 22:20:07.969464 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:07.969561 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":129,"threads_started":1}
I20260811 22:20:08.116976 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:08.116409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:52819)
0811 22:20:08.116733 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:08.116801 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":132,"threads_started":1}
I20260811 22:20:08.320688 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:08.320209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:36245)
0811 22:20:08.320520 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:08.320583 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260811 22:20:08.971011 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:08.970553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:43389)
0811 22:20:08.970811 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:08.970877 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260811 22:20:09.088802 4442 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:09.089075 4442 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:09.089581 4442 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
W20260811 22:20:09.090451 3969 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
W20260811 22:20:09.090610 3966 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:20:09.090693 3966 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37e6577157944daaa74f47dae2748554; no voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257
I20260811 22:20:09.090889 4442 raft_consensus.cc:2726] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:20:09.118526 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:09.117995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:41075)
0811 22:20:09.118356 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:09.118421 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":105,"threads_started":1}
I20260811 22:20:09.322333 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:09.321773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:55741)
0811 22:20:09.322160 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:09.322227 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":90,"threads_started":1}
I20260811 22:20:09.972723 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:09.972109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:38001)
0811 22:20:09.972487 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:09.972579 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":85,"threads_started":1}
I20260811 22:20:10.120149 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:10.119661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:39047)
0811 22:20:10.119983 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:10.120049 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":114,"threads_started":1}
I20260811 22:20:10.323900 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:10.323310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:53767)
0811 22:20:10.323654 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:10.323722 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260811 22:20:10.974296 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:10.973793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:53423)
0811 22:20:10.974125 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:10.974195 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260811 22:20:11.121719 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:11.121178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:59373)
0811 22:20:11.121537 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:11.121611 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":103,"threads_started":1}
I20260811 22:20:11.325541 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:11.324993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:33845)
0811 22:20:11.325359 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:11.325437 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":93,"threads_started":1}
I20260811 22:20:11.975857 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:11.975319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:43325)
0811 22:20:11.975624 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:11.975741 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260811 22:20:12.123129 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:12.122645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:45881)
0811 22:20:12.122956 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:12.123023 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":85,"threads_started":1}
I20260811 22:20:12.327179 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:12.326651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:42007)
0811 22:20:12.326985 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:12.327053 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":125,"threads_started":1}
I20260811 22:20:12.977476 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:12.976961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:52187)
0811 22:20:12.977273 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:12.977344 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260811 22:20:13.124635 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:13.124117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:39967)
0811 22:20:13.124454 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:13.124526 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":104,"threads_started":1}
I20260811 22:20:13.328744 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:13.328273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:56267)
0811 22:20:13.328569 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:13.328642 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260811 22:20:13.573678 4461 raft_consensus.cc:482] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:13.573873 4461 raft_consensus.cc:504] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c909e93b3b41ee8b1b42fd3cbc3b10" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 32853 } } peers { permanent_uuid: "39ef057a72ad431ab579cfde3d6d0257" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38705 } } peers { permanent_uuid: "37e6577157944daaa74f47dae2748554" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 40779 } }
I20260811 22:20:13.574396 4461 leader_election.cc:288] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853), 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705)
W20260811 22:20:13.575244 3969 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c909e93b3b41ee8b1b42fd3cbc3b10 (127.17.198.2:32853): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
W20260811 22:20:13.575418 3966 leader_election.cc:341] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39ef057a72ad431ab579cfde3d6d0257 (127.17.198.3:38705): Not found: Tablet not found: 789c3cfb9f0f4c0192681dd45a3ad4ac
I20260811 22:20:13.575501 3966 leader_election.cc:302] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37e6577157944daaa74f47dae2748554; no voters: 08c909e93b3b41ee8b1b42fd3cbc3b10, 39ef057a72ad431ab579cfde3d6d0257
I20260811 22:20:13.575652 4461 raft_consensus.cc:2726] T 789c3cfb9f0f4c0192681dd45a3ad4ac P 37e6577157944daaa74f47dae2748554 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:20:13.979166 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:13.978614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:49181)
0811 22:20:13.979003 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:13.979066 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":135,"threads_started":1}
I20260811 22:20:14.126125 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:14.125533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:42449)
0811 22:20:14.125939 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:14.126016 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":131,"threads_started":1}
I20260811 22:20:14.330149 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:14.329689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:42613)
0811 22:20:14.329990 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:14.330050 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260811 22:20:14.980775 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:14.980092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:38397)
0811 22:20:14.980608 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:14.980674 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":317,"threads_started":1}
I20260811 22:20:15.127579 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:15.127097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:52533)
0811 22:20:15.127407 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:15.127475 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260811 22:20:15.331393 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:15.330949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:49447)
0811 22:20:15.331213 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:15.331273 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260811 22:20:15.982234 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:15.981741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:34859)
0811 22:20:15.982059 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:15.982129 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260811 22:20:16.129184 4470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:16.128640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:54551)
0811 22:20:16.128981 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:16.129045 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":104,"threads_started":1}
I20260811 22:20:16.333031 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:16.332324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.3:52305)
0811 22:20:16.332672 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:16.332922 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":91,"threads_started":1}
I20260811 22:20:16.983736 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:16.983256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.1:52325)
0811 22:20:16.983554 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:16.983621 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260811 22:20:17.130690 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:17.130211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:37875 (local address 127.17.198.2:43461)
0811 22:20:17.130513 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:17.130584 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.62:37875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260811 22:20:17.313271 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 3904
I20260811 22:20:17.322372 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4113
I20260811 22:20:17.330089 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4259
2026-08-11T22:20:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59677 ms)
[----------] 30 tests from SecurityITest (346847 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:17 dist-test-slave-77v9 krb5kdc[4477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:17 dist-test-slave-77v9 krb5kdc[4477](info): set up 2 sockets
Aug 11 22:20:17 dist-test-slave-77v9 krb5kdc[4477](info): commencing operation
krb5kdc: starting...
W20260811 22:20:18.497203 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:18 dist-test-slave-77v9 krb5kdc[4477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:18.645790 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:38385
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:38385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:18.761909 4490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:18.762262 4490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:18.762351 4490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:18.766510 4490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:18.766635 4490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:18.766673 4490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:18.766718 4490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:18.766779 4490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:18.772647 4490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:38385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:18.773908 4490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:18.775018 4490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:18.784391 4490 server_base.cc:1034] running on GCE node
W20260811 22:20:18.784454 4497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:18.784454 4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:18.784663 4496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:18.785378 4490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:18.786702 4490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:18.787917 4490 hybrid_clock.cc:648] HybridClock initialized: now 1786486818787902 us; error 67 us; skew 500 ppm
Aug 11 22:20:18 dist-test-slave-77v9 krb5kdc[4477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486818, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:18.791209 4490 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:18.792449 4490 webserver.cc:460] Webserver started at http://127.17.198.62:35773/ using document root <none> and password file <none>
I20260811 22:20:18.793205 4490 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:18.793286 4490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:18.793569 4490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:18.796023 4490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "fb8e8d0a8354454288af8c298f16f8de"
format_stamp: "Formatted at 2026-08-11 22:20:18 on dist-test-slave-77v9"
server_key: "20ab132aee83899649884d31230f38d2"
server_key_iv: "014c94cae3961ecbcb19676adbc2c601"
server_key_version: "encryptionkey@0"
I20260811 22:20:18.796764 4490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "fb8e8d0a8354454288af8c298f16f8de"
format_stamp: "Formatted at 2026-08-11 22:20:18 on dist-test-slave-77v9"
server_key: "20ab132aee83899649884d31230f38d2"
server_key_iv: "014c94cae3961ecbcb19676adbc2c601"
server_key_version: "encryptionkey@0"
I20260811 22:20:18.801419 4490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:20:18.804353 4510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:18.805970 4490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:18.806161 4490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "fb8e8d0a8354454288af8c298f16f8de"
format_stamp: "Formatted at 2026-08-11 22:20:18 on dist-test-slave-77v9"
server_key: "20ab132aee83899649884d31230f38d2"
server_key_iv: "014c94cae3961ecbcb19676adbc2c601"
server_key_version: "encryptionkey@0"
I20260811 22:20:18.806313 4490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:18.825899 4490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:18.830107 4490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:18.830374 4490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:18.838321 4490 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:38385
I20260811 22:20:18.838399 4569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:38385 every 8 connection(s)
I20260811 22:20:18.839388 4490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:18.842425 4570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:18.844233 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4490
I20260811 22:20:18.844350 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:18.844679 18200 external_mini_cluster.cc:1428] Setting key 0a813900c4a9a3bc63a2671b092512f8
I20260811 22:20:18.849833 4570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: Bootstrap starting.
I20260811 22:20:18.852638 4570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:18.853549 4570 log.cc:824] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:20:18 dist-test-slave-77v9 krb5kdc[4477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:18.855818 4570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: No bootstrap required, opened a new log
I20260811 22:20:18.860921 4570 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } }
I20260811 22:20:18.861233 4570 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:18.861299 4570 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8e8d0a8354454288af8c298f16f8de, State: Initialized, Role: FOLLOWER
I20260811 22:20:18.861824 4570 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } }
I20260811 22:20:18.861996 4570 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:18.862013 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:18.846371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.17.198.62:38385)
0811 22:20:18.846876 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:18.846896 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:18.846951 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:20:18.847839 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:18.847867 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:18.848329 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:18.848732 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:18.849595 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:18.851083 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:18.851978 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:18.852695 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:18.852904 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:18.855636 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:18.855642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:18.855659 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:18.855692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:18.858037 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:18.858578 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:18.858588 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:18.858598 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:18.858792 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:18.859380 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:18.859384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:18.859386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:18.860002 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:18.860222 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:18.860544 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 22:20:18.860903 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":143,"threads_started":1}
I20260811 22:20:18.862083 4570 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:18.862177 4570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:18.863492 4570 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } }
I20260811 22:20:18.863885 4570 leader_election.cc:302] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8e8d0a8354454288af8c298f16f8de; no voters:
I20260811 22:20:18.864241 4570 leader_election.cc:288] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:18.864418 4577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:18.864748 4577 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [term 1 LEADER]: Becoming Leader. State: Replica: fb8e8d0a8354454288af8c298f16f8de, State: Running, Role: LEADER
I20260811 22:20:18.865175 4577 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } }
I20260811 22:20:18.865660 4570 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:18.867192 4578 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8e8d0a8354454288af8c298f16f8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } } }
I20260811 22:20:18.867379 4578 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:18.867187 4579 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8e8d0a8354454288af8c298f16f8de. Latest consensus state: current_term: 1 leader_uuid: "fb8e8d0a8354454288af8c298f16f8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8e8d0a8354454288af8c298f16f8de" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 38385 } } }
I20260811 22:20:18.867595 4579 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:18.871341 4593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:18.871455 4593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:18.871529 4589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:18.872279 4589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:18.878203 4589 catalog_manager.cc:1347] Generated new cluster ID: 8d3f66518fca44f5a20f63b362a04467
I20260811 22:20:18.878357 4589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:18.894357 4589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:18.895519 4589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:18.904011 4589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: Generated new TSK 0
I20260811 22:20:18.904860 4589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:20:18.946087 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:19.087388 4600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:19.087728 4600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:19.087836 4600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:19.092952 4600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:19.093036 4600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:19.093161 4600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:19.099566 4600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:19.100796 4600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:19.102001 4600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:19.112730 4605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:19.112841 4600 server_base.cc:1034] running on GCE node
W20260811 22:20:19.112749 4606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:19.112749 4609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:19.113520 4600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:19.114202 4600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:19.115406 4600 hybrid_clock.cc:648] HybridClock initialized: now 1786486819115366 us; error 41 us; skew 500 ppm
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:19.118654 4600 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:20:19.119628 4600 webserver.cc:460] Webserver started at http://127.17.198.1:35271/ using document root <none> and password file <none>
I20260811 22:20:19.120291 4600 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:19.120359 4600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:19.120580 4600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:19.122738 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "508be4f1cb6a4c66a2603ce80544c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "d581a449efd001f0488f98a25f04a998"
server_key_iv: "84b4f096c15b99632391b834eb4dedc8"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.123337 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "508be4f1cb6a4c66a2603ce80544c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "d581a449efd001f0488f98a25f04a998"
server_key_iv: "84b4f096c15b99632391b834eb4dedc8"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.127501 4600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:20:19.130123 4618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.131456 4600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:20:19.131716 4600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "508be4f1cb6a4c66a2603ce80544c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "d581a449efd001f0488f98a25f04a998"
server_key_iv: "84b4f096c15b99632391b834eb4dedc8"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.131850 4600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:19.150731 4600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:19.154259 4600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:19.154505 4600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:19.155180 4600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:19.156149 4600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:19.156227 4600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.156329 4600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:19.156380 4600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.166250 4600 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:43399
I20260811 22:20:19.166270 4751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:43399 every 8 connection(s)
I20260811 22:20:19.167291 4600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:19.175894 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4600
I20260811 22:20:19.176088 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:19.176440 18200 external_mini_cluster.cc:1428] Setting key ffab8e63c5fa2bda62a5b288752e83b2
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:19.184160 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.169432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:59073 (local address 127.17.198.62:38385)
0811 22:20:19.169607 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.169611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.170577 (+ 966us) server_negotiation.cc:407] Connection header received
0811 22:20:19.172108 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.172112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.172164 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.172252 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:19.173892 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.174537 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.176110 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.176413 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.176543 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.179652 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:19.179658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:19.179666 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:19.179693 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:19.181610 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.182319 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.182322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.182323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.182425 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.182845 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.182848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.182849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.183046 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:19.183129 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.183872 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.184019 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:20:19.185571 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.169821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38385 (local address 127.17.198.1:59073)
0811 22:20:19.170394 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.170446 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.171853 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.172490 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.172498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.173007 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:19.173603 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.173619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.174689 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.174692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.175901 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.175916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.176617 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.176623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.176882 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.177672 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:19.177691 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:19.179465 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:19.181746 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.181755 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.181776 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.182207 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.182524 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.182530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.182536 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.182743 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.183169 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:19.183180 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:19.183568 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.183826 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.184244 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":122,"threads_started":1}
I20260811 22:20:19.187098 4752 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38385
I20260811 22:20:19.187386 4752 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:19.187948 4752 heartbeater.cc:511] Master 127.17.198.62:38385 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:20:19.189771 4528 ts_manager.cc:194] Registered new tserver with Master: 508be4f1cb6a4c66a2603ce80544c4ea (127.17.198.1:43399)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:20:19.191264 4528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:59073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:20:19.210345 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:19.331714 4759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:19.332072 4759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:19.332155 4759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:19.336706 4759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:19.336884 4759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:19.336997 4759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:19.342730 4759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:19.344121 4759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:19.345415 4759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:19.357172 4759 server_base.cc:1034] running on GCE node
W20260811 22:20:19.357146 4765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:19.357128 4767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:19.357462 4764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:19.358044 4759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:19.358896 4759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:19.360139 4759 hybrid_clock.cc:648] HybridClock initialized: now 1786486819360118 us; error 43 us; skew 500 ppm
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:19.363843 4759 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:20:19.365162 4759 webserver.cc:460] Webserver started at http://127.17.198.2:45411/ using document root <none> and password file <none>
I20260811 22:20:19.365914 4759 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:19.366037 4759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:19.366322 4759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:19.369014 4759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "c84357edc4f3498e88939174b62ee551"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "f810b2c0ee7c80c4fbe881132c9f05e1"
server_key_iv: "acbafee06485796220d23328acfe9a5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.369706 4759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "c84357edc4f3498e88939174b62ee551"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "f810b2c0ee7c80c4fbe881132c9f05e1"
server_key_iv: "acbafee06485796220d23328acfe9a5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.374622 4759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:20:19.377987 4775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.379489 4759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:20:19.379637 4759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "c84357edc4f3498e88939174b62ee551"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "f810b2c0ee7c80c4fbe881132c9f05e1"
server_key_iv: "acbafee06485796220d23328acfe9a5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.379751 4759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:19.393487 4759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:19.397586 4759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:19.397768 4759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:19.398341 4759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:19.399284 4759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:19.399339 4759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.399384 4759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:19.399399 4759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.409817 4759 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:33559
I20260811 22:20:19.409900 4904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:33559 every 8 connection(s)
I20260811 22:20:19.410867 4759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:19.419129 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4759
I20260811 22:20:19.419284 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:19.419592 18200 external_mini_cluster.cc:1428] Setting key d23a98eac456aaeed1c2ab3906b52fcb
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:19.427675 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.413227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:52359 (local address 127.17.198.62:38385)
0811 22:20:19.413429 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.413438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.414305 (+ 867us) server_negotiation.cc:407] Connection header received
0811 22:20:19.415910 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.415913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.415971 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.416082 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:19.417679 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.418268 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.420051 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.420412 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.420574 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.423316 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:19.423320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:19.423323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:19.423361 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:19.424994 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.425569 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.425573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.425575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.425665 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.426037 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.426040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.426042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.426242 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:19.426930 (+ 688us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.427155 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.427372 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:20:19.428653 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.413486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38385 (local address 127.17.198.2:52359)
0811 22:20:19.414108 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.414162 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.415630 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.416273 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.416299 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.416736 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:19.417461 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.417492 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.418455 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.418458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.419839 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.419853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.420606 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.420609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.420771 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.421546 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:19.421568 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:19.423122 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:19.425126 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.425132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.425151 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.425449 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.425762 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.425766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.425768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.425925 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.426375 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:19.426384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:19.426760 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.426981 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.427421 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":147,"threads_started":1}
I20260811 22:20:19.430346 4905 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38385
I20260811 22:20:19.430711 4905 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:19.431480 4905 heartbeater.cc:511] Master 127.17.198.62:38385 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:20:19.432992 4528 ts_manager.cc:194] Registered new tserver with Master: c84357edc4f3498e88939174b62ee551 (127.17.198.2:33559)
I20260811 22:20:19.433759 4528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:52359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:20:19.457855 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:19.578555 4912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:19.578971 4912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:19.579075 4912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:19.584272 4912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:19.584416 4912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:19.584533 4912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:19.591264 4912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:19.592561 4912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:19.593824 4912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:19.604570 4920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:19.604578 4918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:19.604627 4917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:19.605157 4912 server_base.cc:1034] running on GCE node
I20260811 22:20:19.605633 4912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:19.606364 4912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:19.607579 4912 hybrid_clock.cc:648] HybridClock initialized: now 1786486819607557 us; error 34 us; skew 500 ppm
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:19.610813 4912 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:20:19.611898 4912 webserver.cc:460] Webserver started at http://127.17.198.3:44835/ using document root <none> and password file <none>
I20260811 22:20:19.612534 4912 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:19.612602 4912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:19.612927 4912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:19.615146 4912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "21257e439c484fd28e67092d80e6a33c"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "969baacb831a3ddce4dfddd9a2f8205e"
server_key_iv: "90246bc584dff7346d2df97e42725a69"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.615772 4912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "21257e439c484fd28e67092d80e6a33c"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "969baacb831a3ddce4dfddd9a2f8205e"
server_key_iv: "90246bc584dff7346d2df97e42725a69"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.619861 4912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:20:19.622502 4929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.623692 4912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:20:19.623842 4912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "21257e439c484fd28e67092d80e6a33c"
format_stamp: "Formatted at 2026-08-11 22:20:19 on dist-test-slave-77v9"
server_key: "969baacb831a3ddce4dfddd9a2f8205e"
server_key_iv: "90246bc584dff7346d2df97e42725a69"
server_key_version: "encryptionkey@0"
I20260811 22:20:19.623967 4912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:19.645614 4912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:19.649631 4912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:19.649955 4912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:19.650635 4912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:19.651594 4912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:19.651675 4912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.651763 4912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:19.651820 4912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:19.662364 4912 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:40991
I20260811 22:20:19.662400 5058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:40991 every 8 connection(s)
I20260811 22:20:19.663425 4912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:19.667088 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 4912
I20260811 22:20:19.667250 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:19.667636 18200 external_mini_cluster.cc:1428] Setting key bcb180e1a93017f6cef5f7f388d20a74
Aug 11 22:20:19 dist-test-slave-77v9 krb5kdc[4477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:19.680114 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.665653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:35011 (local address 127.17.198.62:38385)
0811 22:20:19.665818 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.665824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.666985 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 22:20:19.668802 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.668850 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.668923 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.669022 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:19.670630 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.671221 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.672515 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.672771 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.672973 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.675959 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:19.675962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:19.675964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:19.675996 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:19.677748 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.678367 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.678372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.678374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.678467 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.678872 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.678875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.678877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.679093 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:19.679204 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.679784 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.679931 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:20:19.681175 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.666148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:38385 (local address 127.17.198.3:35011)
0811 22:20:19.666784 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.666852 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.668509 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.669270 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.669279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.669830 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:19.670472 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.670489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.671387 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.671398 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.672374 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.672383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.673020 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.673025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.673226 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.673940 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:19.673958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:19.675735 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:19.677911 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.677919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.677937 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.678244 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.678576 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:19.678579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:19.678582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:19.678749 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:19.679219 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:19.679228 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:19.679509 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.679720 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.680116 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":136,"threads_started":1}
I20260811 22:20:19.682463 5059 heartbeater.cc:347] Connected to a master server at 127.17.198.62:38385
I20260811 22:20:19.682765 5059 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:19.683307 5059 heartbeater.cc:511] Master 127.17.198.62:38385 requested a full tablet report, sending...
I20260811 22:20:19.684535 4528 ts_manager.cc:194] Registered new tserver with Master: 21257e439c484fd28e67092d80e6a33c (127.17.198.3:40991)
I20260811 22:20:19.685300 4528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:35011
I20260811 22:20:19.695227 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:19.705799 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.697544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.17.198.62:38385)
0811 22:20:19.697715 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.697721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.697924 (+ 203us) server_negotiation.cc:407] Connection header received
0811 22:20:19.698114 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.698117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.698162 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.698251 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:19.699039 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.699585 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.700489 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.700717 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.700855 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.702370 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:19.702373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:19.702375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:19.702402 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:19.703901 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.704368 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.704378 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.704380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.704459 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:19.704749 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:19.704753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:19.704754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:19.705077 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:19.705228 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.705497 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.705626 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:20:19.709221 4528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:19.712857 4528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:19.729179 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.722441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43399 (local address 127.0.0.1:50222)
0811 22:20:19.723232 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.723296 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.723460 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.724226 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.724232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.724255 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:19.724424 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.724433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.725738 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.725744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.727163 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.727178 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.728562 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.728566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.728676 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.728714 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.728885 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.729028 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":90,"threads_started":1}
I20260811 22:20:19.729192 5072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.722655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.17.198.1:43399)
0811 22:20:19.723031 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.723038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.723629 (+ 591us) server_negotiation.cc:407] Connection header received
0811 22:20:19.723724 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.723729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.724044 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.724227 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:19.724550 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.725627 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.727325 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.728406 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.728579 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.728743 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.728921 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.729023 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":138,"threads_started":1}
I20260811 22:20:19.729574 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.722678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:33559 (local address 127.0.0.1:36776)
0811 22:20:19.724000 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.724015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.724099 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.724684 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.724687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.724706 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:19.724800 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.724806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.726413 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.726426 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.727818 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.727830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.729127 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.729131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.729246 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.729280 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.729334 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.729406 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"mutex_wait_us":1,"thread_start_us":131,"threads_started":1}
I20260811 22:20:19.730371 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.723267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:40991 (local address 127.0.0.1:33152)
0811 22:20:19.724311 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.724324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.724399 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.725072 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.725075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.725094 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:19.725178 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.725183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.726649 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.726652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.728028 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.728038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.729787 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.729791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.730037 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.730061 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.730147 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.730222 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":27,"thread_start_us":116,"threads_started":1}
I20260811 22:20:19.730394 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.723748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.17.198.2:33559)
0811 22:20:19.724201 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.724214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.724237 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:20:19.724314 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.724319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.724526 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.724696 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:19.724924 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.726247 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.727986 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.728968 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.729930 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.730079 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.730137 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.730235 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":104,"threads_started":1}
I20260811 22:20:19.730902 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.723463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.17.198.3:40991)
0811 22:20:19.724564 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.724580 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.724606 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:20:19.724685 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.724690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.724910 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.725088 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:19.725273 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.726469 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.728170 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.729187 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.729286 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.729709 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.730647 (+ 938us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.730752 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":178,"threads_started":1}
I20260811 22:20:19.733111 4673 tablet_service.cc:1467] Processing CreateTablet for tablet 74d1b94d46db4626823a3830bfa28a98 (DEFAULT_TABLE table=test-table [id=e293287312be46289995032bfc71bc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:19.733109 4828 tablet_service.cc:1467] Processing CreateTablet for tablet 74d1b94d46db4626823a3830bfa28a98 (DEFAULT_TABLE table=test-table [id=e293287312be46289995032bfc71bc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:19.733451 4981 tablet_service.cc:1467] Processing CreateTablet for tablet 74d1b94d46db4626823a3830bfa28a98 (DEFAULT_TABLE table=test-table [id=e293287312be46289995032bfc71bc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:19.734309 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74d1b94d46db4626823a3830bfa28a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:19.734314 4981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74d1b94d46db4626823a3830bfa28a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:19.734309 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74d1b94d46db4626823a3830bfa28a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:19.741720 5079 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: Bootstrap starting.
I20260811 22:20:19.742561 5080 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Bootstrap starting.
I20260811 22:20:19.743225 5081 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Bootstrap starting.
I20260811 22:20:19.744544 5080 tablet_bootstrap.cc:654] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:19.744632 5079 tablet_bootstrap.cc:654] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:19.745692 5079 log.cc:824] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:19.745996 5080 log.cc:824] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:19.746037 5081 tablet_bootstrap.cc:654] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:19.747013 5081 log.cc:824] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:19.748158 5080 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: No bootstrap required, opened a new log
I20260811 22:20:19.748158 5079 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: No bootstrap required, opened a new log
I20260811 22:20:19.748374 5080 ts_tablet_manager.cc:1397] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:20:19.748374 5079 ts_tablet_manager.cc:1397] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:20:19.748955 5081 tablet_bootstrap.cc:492] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: No bootstrap required, opened a new log
I20260811 22:20:19.749163 5081 ts_tablet_manager.cc:1397] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 22:20:19.752439 5081 raft_consensus.cc:348] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.752799 5081 raft_consensus.cc:374] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:19.752899 5081 raft_consensus.cc:729] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84357edc4f3498e88939174b62ee551, State: Initialized, Role: FOLLOWER
I20260811 22:20:19.752660 5080 raft_consensus.cc:348] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.753247 5080 raft_consensus.cc:374] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:19.753297 5080 raft_consensus.cc:729] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21257e439c484fd28e67092d80e6a33c, State: Initialized, Role: FOLLOWER
I20260811 22:20:19.753126 5079 raft_consensus.cc:348] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.753361 5079 raft_consensus.cc:374] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:19.753422 5079 raft_consensus.cc:729] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508be4f1cb6a4c66a2603ce80544c4ea, State: Initialized, Role: FOLLOWER
I20260811 22:20:19.753491 5081 consensus_queue.cc:260] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.753756 5080 consensus_queue.cc:260] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.753813 5079 consensus_queue.cc:260] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.754395 4905 heartbeater.cc:503] Master 127.17.198.62:38385 was elected leader, sending a full tablet report...
I20260811 22:20:19.754760 5081 ts_tablet_manager.cc:1428] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260811 22:20:19.754827 5059 heartbeater.cc:503] Master 127.17.198.62:38385 was elected leader, sending a full tablet report...
I20260811 22:20:19.755023 4752 heartbeater.cc:503] Master 127.17.198.62:38385 was elected leader, sending a full tablet report...
I20260811 22:20:19.755630 5079 ts_tablet_manager.cc:1428] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:20:19.755705 5080 ts_tablet_manager.cc:1428] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:20:19.847954 5085 raft_consensus.cc:482] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:19.848219 5085 raft_consensus.cc:504] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.849736 5085 leader_election.cc:288] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 508be4f1cb6a4c66a2603ce80544c4ea (127.17.198.1:43399), 21257e439c484fd28e67092d80e6a33c (127.17.198.3:40991)
I20260811 22:20:19.855456 5072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.850201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:53377 (local address 127.17.198.1:43399)
0811 22:20:19.850503 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.850507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.850775 (+ 268us) server_negotiation.cc:407] Connection header received
0811 22:20:19.850935 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.850938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.851003 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.851056 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:19.851473 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.851977 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.853991 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.854693 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.854825 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.854886 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.855222 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.855314 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260811 22:20:19.855456 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.850471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:46199 (local address 127.17.198.3:40991)
0811 22:20:19.850595 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.850599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.850615 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:19.850673 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.850677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.850726 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.850824 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:19.851169 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.852046 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.853939 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.854654 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.854825 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.854886 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.855222 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.855308 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:20:19.855477 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.850066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:40991 (local address 127.17.198.2:46199)
0811 22:20:19.850181 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.850198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.850298 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.850862 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.850865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.850882 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:19.851021 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.851032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.852160 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.852163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.853760 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.853772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.854843 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.854847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.855054 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.855082 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.855169 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.855302 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":295}
I20260811 22:20:19.855486 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.850066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43399 (local address 127.17.198.2:53377)
0811 22:20:19.850671 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.850682 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:19.850802 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:19.851213 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:19.851215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:19.851229 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:19.851346 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.851351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.852121 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.852128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.853760 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:19.853772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.854843 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.854847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:19.855027 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.855090 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:20:19.855169 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:20:19.855333 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":357,"thread_start_us":193,"threads_started":1}
I20260811 22:20:19.856367 5004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74d1b94d46db4626823a3830bfa28a98" candidate_uuid: "c84357edc4f3498e88939174b62ee551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21257e439c484fd28e67092d80e6a33c" is_pre_election: true
I20260811 22:20:19.856438 4697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74d1b94d46db4626823a3830bfa28a98" candidate_uuid: "c84357edc4f3498e88939174b62ee551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" is_pre_election: true
I20260811 22:20:19.856741 4697 raft_consensus.cc:2454] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84357edc4f3498e88939174b62ee551 in term 0.
I20260811 22:20:19.856750 5004 raft_consensus.cc:2454] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84357edc4f3498e88939174b62ee551 in term 0.
I20260811 22:20:19.857407 4778 leader_election.cc:302] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21257e439c484fd28e67092d80e6a33c, 508be4f1cb6a4c66a2603ce80544c4ea, c84357edc4f3498e88939174b62ee551; no voters:
I20260811 22:20:19.857770 5085 raft_consensus.cc:2781] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:19.857897 5085 raft_consensus.cc:482] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:19.857997 5085 raft_consensus.cc:3037] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:19.859275 5085 raft_consensus.cc:504] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.859728 5085 leader_election.cc:288] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [CANDIDATE]: Term 1 election: Requested vote from peers 508be4f1cb6a4c66a2603ce80544c4ea (127.17.198.1:43399), 21257e439c484fd28e67092d80e6a33c (127.17.198.3:40991)
I20260811 22:20:19.860345 5004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74d1b94d46db4626823a3830bfa28a98" candidate_uuid: "c84357edc4f3498e88939174b62ee551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21257e439c484fd28e67092d80e6a33c"
I20260811 22:20:19.860376 4697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74d1b94d46db4626823a3830bfa28a98" candidate_uuid: "c84357edc4f3498e88939174b62ee551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508be4f1cb6a4c66a2603ce80544c4ea"
I20260811 22:20:19.860509 5004 raft_consensus.cc:3037] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:19.860527 4697 raft_consensus.cc:3037] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:19.862025 4697 raft_consensus.cc:2454] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84357edc4f3498e88939174b62ee551 in term 1.
I20260811 22:20:19.862025 5004 raft_consensus.cc:2454] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84357edc4f3498e88939174b62ee551 in term 1.
I20260811 22:20:19.862470 4777 leader_election.cc:302] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21257e439c484fd28e67092d80e6a33c, c84357edc4f3498e88939174b62ee551; no voters:
I20260811 22:20:19.862726 5085 raft_consensus.cc:2781] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:19.862946 5085 raft_consensus.cc:686] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 1 LEADER]: Becoming Leader. State: Replica: c84357edc4f3498e88939174b62ee551, State: Running, Role: LEADER
I20260811 22:20:19.863350 5085 consensus_queue.cc:237] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } }
I20260811 22:20:19.867290 4523 catalog_manager.cc:5495] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 reported cstate change: term changed from 0 to 1, leader changed from <none> to c84357edc4f3498e88939174b62ee551 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "c84357edc4f3498e88939174b62ee551" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84357edc4f3498e88939174b62ee551" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 } health_report { overall_health: UNKNOWN } } }
I20260811 22:20:19.887024 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.882449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.17.198.2:33559)
0811 22:20:19.882629 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:19.882635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:19.882652 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:20:19.882938 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:19.882941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:19.883027 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:19.883169 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:19.883536 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.884202 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.885286 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:19.885536 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:19.885664 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:19.886035 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:19.886534 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:19.886692 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:19.886771 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:20:19.886843 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:20:19.893622 5004 raft_consensus.cc:1261] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 1 FOLLOWER]: Refusing update from remote peer c84357edc4f3498e88939174b62ee551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:19.893621 4697 raft_consensus.cc:1261] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea [term 1 FOLLOWER]: Refusing update from remote peer c84357edc4f3498e88939174b62ee551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:19.894529 5091 consensus_queue.cc:1035] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508be4f1cb6a4c66a2603ce80544c4ea" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:19.894748 5085 consensus_queue.cc:1035] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21257e439c484fd28e67092d80e6a33c" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:19.904934 5096 mvcc.cc:205] Tried to move back new op lower bound from 7317450014275567616 to 7317450014163660800. Current Snapshot: MvccSnapshot[applied={T|T < 7317450014275567616}]
I20260811 22:20:19.915483 4526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:19.915731 4526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:19.919039 4526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb8e8d0a8354454288af8c298f16f8de: Sending DeleteTablet for 3 replicas of tablet 74d1b94d46db4626823a3830bfa28a98
I20260811 22:20:19.920271 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet 74d1b94d46db4626823a3830bfa28a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:19 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:50222
I20260811 22:20:19.920274 4981 tablet_service.cc:1514] Processing DeleteTablet for tablet 74d1b94d46db4626823a3830bfa28a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:19 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:33152
I20260811 22:20:19.920388 4828 tablet_service.cc:1514] Processing DeleteTablet for tablet 74d1b94d46db4626823a3830bfa28a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:19 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:36776
I20260811 22:20:19.920969 5104 tablet_replica.cc:330] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: stopping tablet replica
I20260811 22:20:19.921061 5105 tablet_replica.cc:330] T 74d1b94d46db4626823a3830bfa28a98 P 508be4f1cb6a4c66a2603ce80544c4ea: stopping tablet replica
I20260811 22:20:19.921317 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4600
I20260811 22:20:19.922690 5104 raft_consensus.cc:2229] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:19.923136 5104 raft_consensus.cc:2258] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:19.925491 5104 ts_tablet_manager.cc:1905] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:19.928542 5106 tablet_replica.cc:330] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: stopping tablet replica
I20260811 22:20:19.929011 5106 raft_consensus.cc:2229] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:19.929008 5104 ts_tablet_manager.cc:1918] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:19.929122 5104 log.cc:1196] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/74d1b94d46db4626823a3830bfa28a98
I20260811 22:20:19.929492 5106 raft_consensus.cc:2258] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:19.929527 5104 ts_tablet_manager.cc:1939] T 74d1b94d46db4626823a3830bfa28a98 P 21257e439c484fd28e67092d80e6a33c: Deleting consensus metadata
I20260811 22:20:19.934190 4511 catalog_manager.cc:4841] TS 21257e439c484fd28e67092d80e6a33c (127.17.198.3:40991): tablet 74d1b94d46db4626823a3830bfa28a98 (table test-table [id=e293287312be46289995032bfc71bc9e]) successfully deleted
I20260811 22:20:19.936174 5106 ts_tablet_manager.cc:1905] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:19.940312 5106 ts_tablet_manager.cc:1918] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:19.940501 5106 log.cc:1196] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/74d1b94d46db4626823a3830bfa28a98
I20260811 22:20:19.940915 5106 ts_tablet_manager.cc:1939] T 74d1b94d46db4626823a3830bfa28a98 P c84357edc4f3498e88939174b62ee551: Deleting consensus metadata
W20260811 22:20:19.941978 4512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:43399 (error 108)
I20260811 22:20:19.942430 4511 catalog_manager.cc:4841] TS c84357edc4f3498e88939174b62ee551 (127.17.198.2:33559): tablet 74d1b94d46db4626823a3830bfa28a98 (table test-table [id=e293287312be46289995032bfc71bc9e]) successfully deleted
I20260811 22:20:19.942994 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4759
I20260811 22:20:19.943121 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:19.942784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:43399 (local address 127.0.0.1:50224)
0811 22:20:19.942935 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:19.943009 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:20:19.946736 4512 catalog_manager.cc:4584] TS 508be4f1cb6a4c66a2603ce80544c4ea (127.17.198.1:43399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74d1b94d46db4626823a3830bfa28a98: Network error: Client connection negotiation failed: client connection to 127.17.198.1:43399: connect: Connection refused (error 111)
I20260811 22:20:19.951398 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4912
I20260811 22:20:19.960033 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 4490
2026-08-11T22:20:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2630 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:19Z Disabled control of system clock
I20260811 22:20:19.998654 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39503
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:20.122640 5111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:20.123072 5111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:20.123116 5111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:20.127445 5111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:20.127526 5111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:20.127542 5111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:20.127560 5111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:20.127575 5111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:20.132624 5111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:39503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:20.133903 5111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:20.135336 5111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:20.144730 5117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.144749 5111 server_base.cc:1034] running on GCE node
W20260811 22:20:20.144737 5116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.144970 5120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.145624 5111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:20.146914 5111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:20.148151 5111 hybrid_clock.cc:648] HybridClock initialized: now 1786486820148129 us; error 41 us; skew 500 ppm
I20260811 22:20:20.150460 5111 webserver.cc:460] Webserver started at http://127.17.198.62:38061/ using document root <none> and password file <none>
I20260811 22:20:20.151104 5111 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:20.151211 5111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:20.151469 5111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:20.153738 5111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c160d75ac902442ea64e20cefa3541c5"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "03701f3a482e266796fda8b940e170c7"
server_key_iv: "f0b6c4a857a69d69be7dbd04dbb85b5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.154376 5111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c160d75ac902442ea64e20cefa3541c5"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "03701f3a482e266796fda8b940e170c7"
server_key_iv: "f0b6c4a857a69d69be7dbd04dbb85b5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.158718 5111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:20.161953 5126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.163386 5111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:20:20.163559 5111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c160d75ac902442ea64e20cefa3541c5"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "03701f3a482e266796fda8b940e170c7"
server_key_iv: "f0b6c4a857a69d69be7dbd04dbb85b5c"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.163698 5111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:20.177675 5111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:20.178493 5111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:20.178730 5111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:20.186818 5111 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:39503
I20260811 22:20:20.186868 5188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:39503 every 8 connection(s)
I20260811 22:20:20.187887 5111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:20.191255 5189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:20.197896 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5111
I20260811 22:20:20.198076 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:20.198165 5189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: Bootstrap starting.
I20260811 22:20:20.198379 18200 external_mini_cluster.cc:1428] Setting key 295a351062040c4dbcd782936acb5aed
I20260811 22:20:20.201086 5189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:20.201913 5189 log.cc:824] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:20.203877 5189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: No bootstrap required, opened a new log
I20260811 22:20:20.207873 5189 raft_consensus.cc:348] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } }
I20260811 22:20:20.208143 5189 raft_consensus.cc:374] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:20.208228 5189 raft_consensus.cc:729] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c160d75ac902442ea64e20cefa3541c5, State: Initialized, Role: FOLLOWER
I20260811 22:20:20.208266 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.199919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.17.198.62:39503)
0811 22:20:20.200610 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.200624 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.200661 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:20:20.201489 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.201509 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.201930 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.202317 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:20.202763 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.203737 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.204724 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.205279 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.205465 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.206388 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:20.206393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:20.206406 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:20.206823 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:20.206882 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:20.206902 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:20.207025 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.207174 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.207445 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":135,"threads_started":1}
I20260811 22:20:20.208765 5189 consensus_queue.cc:260] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } }
I20260811 22:20:20.208966 5189 raft_consensus.cc:388] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:20.209025 5189 raft_consensus.cc:482] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:20.209108 5189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:20.210144 5189 raft_consensus.cc:504] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } }
I20260811 22:20:20.210541 5189 leader_election.cc:302] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c160d75ac902442ea64e20cefa3541c5; no voters:
I20260811 22:20:20.210907 5189 leader_election.cc:288] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:20.211061 5194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:20.211413 5194 raft_consensus.cc:686] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [term 1 LEADER]: Becoming Leader. State: Replica: c160d75ac902442ea64e20cefa3541c5, State: Running, Role: LEADER
I20260811 22:20:20.211740 5194 consensus_queue.cc:237] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } }
I20260811 22:20:20.212203 5189 sys_catalog.cc:564] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:20.213588 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c160d75ac902442ea64e20cefa3541c5. Latest consensus state: current_term: 1 leader_uuid: "c160d75ac902442ea64e20cefa3541c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } } }
I20260811 22:20:20.213737 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:20.214128 5195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c160d75ac902442ea64e20cefa3541c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c160d75ac902442ea64e20cefa3541c5" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 39503 } } }
I20260811 22:20:20.214262 5195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5 [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:20.217334 5209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:20.217435 5209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:20.217496 5205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:20.218372 5205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:20.224519 5205 catalog_manager.cc:1347] Generated new cluster ID: 79cdb392bc38466daad412966665d2ac
I20260811 22:20:20.224615 5205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:20.241012 5205 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:20.242040 5205 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:20.254971 5205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: Generated new TSK 0
I20260811 22:20:20.255785 5205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:20.260345 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--builtin_ntp_servers=127.17.198.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:20.379326 5213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:20.379628 5213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:20.379669 5213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:20.383904 5213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:20.383983 5213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:20.384043 5213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:20.389168 5213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:20.390396 5213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:20.391855 5213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:20.402139 5219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.402129 5218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.402134 5222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.402669 5213 server_base.cc:1034] running on GCE node
I20260811 22:20:20.403172 5213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:20.403847 5213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:20.405045 5213 hybrid_clock.cc:648] HybridClock initialized: now 1786486820405028 us; error 28 us; skew 500 ppm
I20260811 22:20:20.407127 5213 webserver.cc:460] Webserver started at http://127.17.198.1:38149/ using document root <none> and password file <none>
I20260811 22:20:20.407766 5213 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:20.407866 5213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:20.408107 5213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:20.410286 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "2221918fd51846d8a7ca249a8e9998be"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "c921af3c20ad0dee8b50be8fd2cb74c1"
server_key_iv: "16c2e50f7072ad4d6caf3a1886406761"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.410873 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "2221918fd51846d8a7ca249a8e9998be"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "c921af3c20ad0dee8b50be8fd2cb74c1"
server_key_iv: "16c2e50f7072ad4d6caf3a1886406761"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.414804 5213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:20.417299 5229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.418562 5213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:20:20.418704 5213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "2221918fd51846d8a7ca249a8e9998be"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "c921af3c20ad0dee8b50be8fd2cb74c1"
server_key_iv: "16c2e50f7072ad4d6caf3a1886406761"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.418830 5213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:20.433295 5213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:20.434015 5213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:20.434221 5213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:20.434823 5213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:20.435823 5213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:20.435895 5213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.435971 5213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:20.436017 5213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.444926 5213 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:33257
I20260811 22:20:20.444967 5362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:33257 every 8 connection(s)
I20260811 22:20:20.445909 5213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:20.448246 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5213
I20260811 22:20:20.448413 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:20.448726 18200 external_mini_cluster.cc:1428] Setting key e30b85160a8727c4a17a94a5f8e15eeb
I20260811 22:20:20.451049 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--builtin_ntp_servers=127.17.198.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:20.457420 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.447915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:45021 (local address 127.17.198.62:39503)
0811 22:20:20.448089 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.448093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.449604 (+ 1511us) server_negotiation.cc:407] Connection header received
0811 22:20:20.451297 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.451303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.451381 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.451485 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:20.452498 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.453294 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.454917 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.455321 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.455455 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.456739 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:20.456745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:20.456752 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:20.456771 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:20.456799 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:20.456846 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:20.456950 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.457110 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.457193 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:20:20.458244 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.448561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39503 (local address 127.17.198.1:45021)
0811 22:20:20.449434 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.449492 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.450998 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.451674 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.451681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.451774 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:20.452327 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.452341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.453453 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.453459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.454758 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.454768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.455506 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.455509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.455671 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.456437 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:20.456439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:20.456481 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:20.456554 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:20.456961 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:20.456969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:20.456974 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.457082 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.457332 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":187,"threads_started":1}
I20260811 22:20:20.459640 5363 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39503
I20260811 22:20:20.459905 5363 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:20.460481 5363 heartbeater.cc:511] Master 127.17.198.62:39503 requested a full tablet report, sending...
I20260811 22:20:20.462105 5146 ts_manager.cc:194] Registered new tserver with Master: 2221918fd51846d8a7ca249a8e9998be (127.17.198.1:33257)
I20260811 22:20:20.463270 5146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.1:45021
W20260811 22:20:20.567600 5367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:20.567905 5367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:20.567946 5367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:20.572247 5367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:20.572332 5367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:20.572461 5367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:20.577804 5367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:20.579051 5367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:20.580502 5367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:20.590869 5376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.590901 5378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.591086 5367 server_base.cc:1034] running on GCE node
W20260811 22:20:20.590924 5375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.591620 5367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:20.592490 5367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:20.593743 5367 hybrid_clock.cc:648] HybridClock initialized: now 1786486820593710 us; error 48 us; skew 500 ppm
I20260811 22:20:20.596204 5367 webserver.cc:460] Webserver started at http://127.17.198.2:33295/ using document root <none> and password file <none>
I20260811 22:20:20.597008 5367 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:20.597132 5367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:20.597426 5367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:20.599932 5367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "8ade0cf4afa84fb1b2049139c0d1e66d"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fd279f211a5d87c510d3c2606dc5d6b0"
server_key_iv: "932834e553c953a1a542cd82586023f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.600683 5367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "8ade0cf4afa84fb1b2049139c0d1e66d"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fd279f211a5d87c510d3c2606dc5d6b0"
server_key_iv: "932834e553c953a1a542cd82586023f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.605144 5367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:20:20.607849 5384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.609259 5367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:20.609432 5367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "8ade0cf4afa84fb1b2049139c0d1e66d"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fd279f211a5d87c510d3c2606dc5d6b0"
server_key_iv: "932834e553c953a1a542cd82586023f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.609573 5367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:20.620950 5367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:20.621707 5367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:20.621937 5367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:20.622568 5367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:20.623587 5367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:20.623636 5367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.623677 5367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:20.623701 5367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.633733 5367 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:46461
I20260811 22:20:20.633764 5511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:46461 every 8 connection(s)
I20260811 22:20:20.634701 5367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:20.639793 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5367
I20260811 22:20:20.639921 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:20.640239 18200 external_mini_cluster.cc:1428] Setting key d70db50b3077adef3af9e84a47effc9a
I20260811 22:20:20.642524 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--builtin_ntp_servers=127.17.198.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:20.645403 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.636651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:50043 (local address 127.17.198.62:39503)
0811 22:20:20.636837 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.636843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.637964 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 22:20:20.639363 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.639366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.639414 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.639510 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:20.640773 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.641489 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.642979 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.643384 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.643523 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.644909 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:20.644912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:20.644914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:20.644928 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:20.644951 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:20.644960 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:20.645024 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.645225 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.645280 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:20:20.646533 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.637119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39503 (local address 127.17.198.2:50043)
0811 22:20:20.637792 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.637844 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.639142 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.639733 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.639743 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.639854 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:20.640593 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.640613 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.641684 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.641691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.642853 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.642863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.643562 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.643565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.643721 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.644514 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:20.644519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:20.644587 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:20.644682 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:20.645052 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:20.645064 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:20.645072 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.645223 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.645564 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":114,"threads_started":1}
I20260811 22:20:20.647871 5512 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39503
I20260811 22:20:20.648164 5512 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:20.648955 5512 heartbeater.cc:511] Master 127.17.198.62:39503 requested a full tablet report, sending...
I20260811 22:20:20.650316 5146 ts_manager.cc:194] Registered new tserver with Master: 8ade0cf4afa84fb1b2049139c0d1e66d (127.17.198.2:46461)
I20260811 22:20:20.651077 5146 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.2:50043
W20260811 22:20:20.762946 5515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:20.763249 5515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:20.763291 5515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:20.767529 5515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:20.767607 5515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:20.767700 5515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:20.772982 5515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:20.774220 5515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:20.775684 5515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:20.786761 5520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.786754 5523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:20.786749 5521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:20.787321 5515 server_base.cc:1034] running on GCE node
I20260811 22:20:20.787765 5515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:20.788496 5515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:20.789733 5515 hybrid_clock.cc:648] HybridClock initialized: now 1786486820789712 us; error 45 us; skew 500 ppm
I20260811 22:20:20.792052 5515 webserver.cc:460] Webserver started at http://127.17.198.3:43213/ using document root <none> and password file <none>
I20260811 22:20:20.792737 5515 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:20.792943 5515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:20.793231 5515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:20.795686 5515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "db8a873d863947998b8dd43d92e11546"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fc490e5b0d9e11c75300ef9c4814e8b5"
server_key_iv: "bee2d5ad6189d515fe69a7be6879c7e8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.796365 5515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "db8a873d863947998b8dd43d92e11546"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fc490e5b0d9e11c75300ef9c4814e8b5"
server_key_iv: "bee2d5ad6189d515fe69a7be6879c7e8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.801008 5515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:20:20.803754 5529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.805177 5515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:20.805289 5515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "db8a873d863947998b8dd43d92e11546"
format_stamp: "Formatted at 2026-08-11 22:20:20 on dist-test-slave-77v9"
server_key: "fc490e5b0d9e11c75300ef9c4814e8b5"
server_key_iv: "bee2d5ad6189d515fe69a7be6879c7e8"
server_key_version: "encryptionkey@0"
I20260811 22:20:20.805388 5515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:20.827069 5515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:20.827847 5515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:20.828086 5515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:20.828738 5515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:20.829787 5515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:20.829840 5515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.829880 5515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:20.829942 5515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:20.839779 5515 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:40779
I20260811 22:20:20.839879 5655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:40779 every 8 connection(s)
I20260811 22:20:20.840873 5515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:20.841521 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5515
I20260811 22:20:20.841656 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:20.841930 18200 external_mini_cluster.cc:1428] Setting key d663247127b43bed792ac5b6623ec29f
I20260811 22:20:20.851594 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.843099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:39179 (local address 127.17.198.62:39503)
0811 22:20:20.843221 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.843230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.844646 (+ 1416us) server_negotiation.cc:407] Connection header received
0811 22:20:20.846245 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.846248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.846295 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.846366 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:20.847325 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.847860 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.849255 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.849493 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.849579 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.851006 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:20.851010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:20.851012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:20.851026 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:20.851046 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:20.851054 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:20.851126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.851395 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.851462 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 22:20:20.852741 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.843643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:39503 (local address 127.17.198.3:39179)
0811 22:20:20.844474 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.844518 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.846055 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.846531 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.846538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.846621 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:20.847198 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.847212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.848021 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.848025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.849111 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.849122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.849671 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.849674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.849909 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.850679 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:20.850681 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:20.850725 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:20.850822 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:20.851182 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:20.851193 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:20.851198 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.851341 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.851639 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":155,"threads_started":1}
I20260811 22:20:20.854240 5657 heartbeater.cc:347] Connected to a master server at 127.17.198.62:39503
I20260811 22:20:20.854579 5657 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:20.855301 5657 heartbeater.cc:511] Master 127.17.198.62:39503 requested a full tablet report, sending...
I20260811 22:20:20.856649 5141 ts_manager.cc:194] Registered new tserver with Master: db8a873d863947998b8dd43d92e11546 (127.17.198.3:40779)
I20260811 22:20:20.857386 5141 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.3:39179
I20260811 22:20:20.869700 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:20.876111 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.871966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.17.198.62:39503)
0811 22:20:20.872108 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.872112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.872230 (+ 118us) server_negotiation.cc:407] Connection header received
0811 22:20:20.872318 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.872321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.872377 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.872460 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:20.872755 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.873477 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.874383 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.874621 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.874713 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.875666 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:20.875669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:20.875672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:20.875686 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:20.875708 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:20.875717 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:20.875790 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.875915 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.875979 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:20:20.879768 5141 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:20.883104 5141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:20.901613 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.893713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:40779 (local address 127.0.0.1:42206)
0811 22:20:20.894168 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.894232 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.894382 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.896407 (+ 2025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.896414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.896443 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:20.896642 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.896654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.898129 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.898132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.899524 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.899535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.900958 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.900962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.901088 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.901136 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.901254 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.901421 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":4,"thread_start_us":108,"threads_started":1}
I20260811 22:20:20.902070 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.895010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46461 (local address 127.0.0.1:46896)
0811 22:20:20.895287 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.895302 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.895407 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.897391 (+ 1984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.897394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.897413 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:20.897540 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.897547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.899083 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.899086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.900196 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.900208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.901312 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.901315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.901407 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.901472 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.901525 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.901585 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":3,"thread_start_us":118,"threads_started":1}
I20260811 22:20:20.902181 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.895390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.17.198.3:40779)
0811 22:20:20.895811 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.895826 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.895855 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:20:20.895962 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.895968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.896213 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.896397 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:20.896836 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.897989 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.899672 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.900530 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.900638 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.900876 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.901871 (+ 995us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.902010 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":196,"threads_started":1}
I20260811 22:20:20.902892 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.895219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.17.198.2:46461)
0811 22:20:20.896921 (+ 1702us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.896934 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.896958 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:20:20.897033 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.897038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.897228 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.897691 (+ 463us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:20.897830 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.898948 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.900364 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.901169 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.902360 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.902591 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.902640 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.902735 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1550,"thread_start_us":142,"threads_started":1}
I20260811 22:20:20.905836 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.894599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:33257 (local address 127.0.0.1:47164)
0811 22:20:20.894916 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:20.894934 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:20.895031 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:20.901223 (+ 6192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:20.901229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:20.901261 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:20.901705 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.901719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.903184 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.903187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.904292 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:20.904303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.905438 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.905445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:20.905550 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.905577 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:20:20.905630 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:20:20.905701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":235,"thread_start_us":127,"threads_started":1}
I20260811 22:20:20.905409 5584 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc (DEFAULT_TABLE table=test-table [id=8ed43f0ddc86420895deeb6e5609d664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:20.905408 5437 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc (DEFAULT_TABLE table=test-table [id=8ed43f0ddc86420895deeb6e5609d664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:20.906656 5673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:20.896015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.17.198.1:33257)
0811 22:20:20.896785 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:20.896799 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:20.896860 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:20:20.896955 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:20.896963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:20.897199 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:20.897391 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:20.901894 (+ 4503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.903037 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.904477 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:20.905272 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:20.906122 (+ 850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:20.906314 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:20.906369 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:20:20.906475 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":117,"threads_started":1}
I20260811 22:20:20.906701 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:20.906709 5437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:20.909629 5288 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc (DEFAULT_TABLE table=test-table [id=8ed43f0ddc86420895deeb6e5609d664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:20.910863 5288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:20.916005 5674 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Bootstrap starting.
I20260811 22:20:20.917173 5675 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Bootstrap starting.
I20260811 22:20:20.918570 5674 tablet_bootstrap.cc:654] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:20.918892 5676 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Bootstrap starting.
I20260811 22:20:20.919319 5675 tablet_bootstrap.cc:654] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:20.919492 5674 log.cc:824] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:20.920209 5675 log.cc:824] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:20.921789 5676 tablet_bootstrap.cc:654] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:20.922612 5674 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: No bootstrap required, opened a new log
I20260811 22:20:20.922607 5676 log.cc:824] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:20.922825 5674 ts_tablet_manager.cc:1397] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:20:20.923592 5675 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: No bootstrap required, opened a new log
I20260811 22:20:20.923834 5675 ts_tablet_manager.cc:1397] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260811 22:20:20.924314 5676 tablet_bootstrap.cc:492] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: No bootstrap required, opened a new log
I20260811 22:20:20.924453 5676 ts_tablet_manager.cc:1397] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 22:20:20.927541 5676 raft_consensus.cc:348] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.927531 5674 raft_consensus.cc:348] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.927846 5676 raft_consensus.cc:374] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:20.927846 5674 raft_consensus.cc:374] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:20.927928 5674 raft_consensus.cc:729] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ade0cf4afa84fb1b2049139c0d1e66d, State: Initialized, Role: FOLLOWER
I20260811 22:20:20.927908 5676 raft_consensus.cc:729] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2221918fd51846d8a7ca249a8e9998be, State: Initialized, Role: FOLLOWER
I20260811 22:20:20.928408 5674 consensus_queue.cc:260] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.928478 5676 consensus_queue.cc:260] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.928507 5675 raft_consensus.cc:348] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.928975 5675 raft_consensus.cc:374] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:20.929067 5675 raft_consensus.cc:729] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8a873d863947998b8dd43d92e11546, State: Initialized, Role: FOLLOWER
I20260811 22:20:20.929168 5363 heartbeater.cc:503] Master 127.17.198.62:39503 was elected leader, sending a full tablet report...
I20260811 22:20:20.929508 5676 ts_tablet_manager.cc:1428] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:20:20.929504 5674 ts_tablet_manager.cc:1428] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:20:20.929667 5675 consensus_queue.cc:260] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:20.929843 5512 heartbeater.cc:503] Master 127.17.198.62:39503 was elected leader, sending a full tablet report...
I20260811 22:20:20.930553 5657 heartbeater.cc:503] Master 127.17.198.62:39503 was elected leader, sending a full tablet report...
I20260811 22:20:20.930675 5675 ts_tablet_manager.cc:1428] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
W20260811 22:20:20.947368 5364 tablet.cc:2369] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:21.048513 5682 raft_consensus.cc:482] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:21.048801 5682 raft_consensus.cc:504] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:21.050136 5682 leader_election.cc:288] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ade0cf4afa84fb1b2049139c0d1e66d (127.17.198.2:46461), 2221918fd51846d8a7ca249a8e9998be (127.17.198.1:33257)
I20260811 22:20:21.054878 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.050501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:34517 (local address 127.17.198.2:46461)
0811 22:20:21.050654 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:21.050658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:21.050674 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:21.050869 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:21.050872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:21.050942 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:21.051029 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:21.051354 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.052102 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.053641 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.054239 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.054373 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:21.054442 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:21.054677 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:20:21.054748 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:20:21.054898 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.050494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:33257 (local address 127.17.198.3:55679)
0811 22:20:21.050890 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:21.050900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:21.051021 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:21.051423 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:21.051425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:21.051443 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:21.051563 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:21.051568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.052474 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.052477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:21.053639 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:21.053646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.054452 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.054456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:21.054541 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:21.054566 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:20:21.054649 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:20:21.054737 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":23,"thread_start_us":129,"threads_started":1}
I20260811 22:20:21.054878 5673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.050568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:55679 (local address 127.17.198.1:33257)
0811 22:20:21.050706 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:21.050711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:21.050994 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:20:21.051151 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:21.051154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:21.051209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:21.051271 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:21.051700 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.052369 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.053754 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.054332 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.054442 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:21.054499 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:21.054677 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 22:20:21.054748 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:20:21.054920 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.050352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46461 (local address 127.17.198.3:34517)
0811 22:20:21.050559 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:21.050577 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:21.050682 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:21.051071 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:21.051075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:21.051091 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:21.051219 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:21.051228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.052232 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.052234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:21.053508 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:21.053518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.054369 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.054372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:21.054516 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:21.054554 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:20:21.054640 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:20:21.054737 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":6}
I20260811 22:20:21.055589 5460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7b36f4f8b44ab2ba69f5e98ce641cc" candidate_uuid: "db8a873d863947998b8dd43d92e11546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" is_pre_election: true
I20260811 22:20:21.055627 5310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7b36f4f8b44ab2ba69f5e98ce641cc" candidate_uuid: "db8a873d863947998b8dd43d92e11546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2221918fd51846d8a7ca249a8e9998be" is_pre_election: true
I20260811 22:20:21.055908 5460 raft_consensus.cc:2454] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8a873d863947998b8dd43d92e11546 in term 0.
I20260811 22:20:21.055923 5310 raft_consensus.cc:2454] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8a873d863947998b8dd43d92e11546 in term 0.
I20260811 22:20:21.056411 5533 leader_election.cc:302] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ade0cf4afa84fb1b2049139c0d1e66d, db8a873d863947998b8dd43d92e11546; no voters:
I20260811 22:20:21.056697 5682 raft_consensus.cc:2781] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:21.056787 5682 raft_consensus.cc:482] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:21.056859 5682 raft_consensus.cc:3037] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:21.057924 5682 raft_consensus.cc:504] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:21.058307 5682 leader_election.cc:288] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [CANDIDATE]: Term 1 election: Requested vote from peers 8ade0cf4afa84fb1b2049139c0d1e66d (127.17.198.2:46461), 2221918fd51846d8a7ca249a8e9998be (127.17.198.1:33257)
I20260811 22:20:21.058655 5460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7b36f4f8b44ab2ba69f5e98ce641cc" candidate_uuid: "db8a873d863947998b8dd43d92e11546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d"
I20260811 22:20:21.058687 5310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7b36f4f8b44ab2ba69f5e98ce641cc" candidate_uuid: "db8a873d863947998b8dd43d92e11546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2221918fd51846d8a7ca249a8e9998be"
I20260811 22:20:21.058773 5460 raft_consensus.cc:3037] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:21.058864 5310 raft_consensus.cc:3037] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:21.059779 5460 raft_consensus.cc:2454] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8a873d863947998b8dd43d92e11546 in term 1.
I20260811 22:20:21.060041 5310 raft_consensus.cc:2454] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8a873d863947998b8dd43d92e11546 in term 1.
I20260811 22:20:21.060087 5533 leader_election.cc:302] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ade0cf4afa84fb1b2049139c0d1e66d, db8a873d863947998b8dd43d92e11546; no voters:
I20260811 22:20:21.060263 5682 raft_consensus.cc:2781] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:21.060509 5682 raft_consensus.cc:686] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 1 LEADER]: Becoming Leader. State: Replica: db8a873d863947998b8dd43d92e11546, State: Running, Role: LEADER
I20260811 22:20:21.060897 5682 consensus_queue.cc:237] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } }
I20260811 22:20:21.064419 5145 catalog_manager.cc:5495] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 reported cstate change: term changed from 0 to 1, leader changed from <none> to db8a873d863947998b8dd43d92e11546 (127.17.198.3). New cstate: current_term: 1 leader_uuid: "db8a873d863947998b8dd43d92e11546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db8a873d863947998b8dd43d92e11546" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 } health_report { overall_health: UNKNOWN } } }
W20260811 22:20:21.092200 5658 tablet.cc:2369] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:21.098099 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.094699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.17.198.3:40779)
0811 22:20:21.094823 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:21.094827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:21.094841 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:20:21.094963 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:21.094966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:21.095024 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:21.095115 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:21.095364 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.095858 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.096730 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:21.096961 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:21.097037 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:21.097343 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:21.097763 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:21.097853 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:21.097920 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 22:20:21.097967 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:20:21.104664 5460 raft_consensus.cc:1261] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 1 FOLLOWER]: Refusing update from remote peer db8a873d863947998b8dd43d92e11546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:21.104763 5310 raft_consensus.cc:1261] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 1 FOLLOWER]: Refusing update from remote peer db8a873d863947998b8dd43d92e11546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:21.105486 5684 consensus_queue.cc:1035] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ade0cf4afa84fb1b2049139c0d1e66d" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:21.105690 5682 consensus_queue.cc:1035] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2221918fd51846d8a7ca249a8e9998be" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:21.112494 5688 mvcc.cc:205] Tried to move back new op lower bound from 7317450019236036608 to 7317450019068289024. Current Snapshot: MvccSnapshot[applied={T|T < 7317450019236036608}]
I20260811 22:20:21.117460 5690 mvcc.cc:205] Tried to move back new op lower bound from 7317450019236036608 to 7317450019068289024. Current Snapshot: MvccSnapshot[applied={T|T < 7317450019236036608}]
I20260811 22:20:21.121907 5145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:21.122102 5145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:21.124617 5145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c160d75ac902442ea64e20cefa3541c5: Sending DeleteTablet for 3 replicas of tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc
I20260811 22:20:21.125422 5584 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:21 UTC) from {username='slave'} at 127.0.0.1:42206
I20260811 22:20:21.125514 5288 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:21 UTC) from {username='slave'} at 127.0.0.1:47164
I20260811 22:20:21.125604 5437 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:21 UTC) from {username='slave'} at 127.0.0.1:46896
I20260811 22:20:21.126077 5698 tablet_replica.cc:330] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: stopping tablet replica
I20260811 22:20:21.126359 5700 tablet_replica.cc:330] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be: stopping tablet replica
I20260811 22:20:21.126415 5698 raft_consensus.cc:2229] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:21.126784 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5213
I20260811 22:20:21.126758 5700 raft_consensus.cc:2229] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 2221918fd51846d8a7ca249a8e9998be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:21.126848 5698 raft_consensus.cc:2258] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:21.128106 5701 tablet_replica.cc:330] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: stopping tablet replica
I20260811 22:20:21.128961 5701 raft_consensus.cc:2229] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:21.129344 5701 raft_consensus.cc:2258] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:21.129715 5698 ts_tablet_manager.cc:1905] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:21.131366 5701 ts_tablet_manager.cc:1905] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:21.132733 5698 ts_tablet_manager.cc:1918] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:21.132872 5698 log.cc:1196] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/1b7b36f4f8b44ab2ba69f5e98ce641cc
I20260811 22:20:21.133252 5698 ts_tablet_manager.cc:1939] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P db8a873d863947998b8dd43d92e11546: Deleting consensus metadata
I20260811 22:20:21.134650 5127 catalog_manager.cc:4841] TS db8a873d863947998b8dd43d92e11546 (127.17.198.3:40779): tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc (table test-table [id=8ed43f0ddc86420895deeb6e5609d664]) successfully deleted
I20260811 22:20:21.135836 5701 ts_tablet_manager.cc:1918] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:21.135955 5701 log.cc:1196] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/1b7b36f4f8b44ab2ba69f5e98ce641cc
I20260811 22:20:21.136297 5701 ts_tablet_manager.cc:1939] T 1b7b36f4f8b44ab2ba69f5e98ce641cc P 8ade0cf4afa84fb1b2049139c0d1e66d: Deleting consensus metadata
I20260811 22:20:21.137773 5130 catalog_manager.cc:4841] TS 8ade0cf4afa84fb1b2049139c0d1e66d (127.17.198.2:46461): tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc (table test-table [id=8ed43f0ddc86420895deeb6e5609d664]) successfully deleted
W20260811 22:20:21.138854 5127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:33257 (error 108)
I20260811 22:20:21.139005 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5367
I20260811 22:20:21.143414 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:21.143067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:33257 (local address 127.0.0.1:47180)
0811 22:20:21.143219 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:21.143299 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:20:21.143628 5127 catalog_manager.cc:4584] TS 2221918fd51846d8a7ca249a8e9998be (127.17.198.1:33257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b7b36f4f8b44ab2ba69f5e98ce641cc: Network error: Client connection negotiation failed: client connection to 127.17.198.1:33257: connect: Connection refused (error 111)
I20260811 22:20:21.151351 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5515
I20260811 22:20:21.162691 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5111
2026-08-11T22:20:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1200 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:21 dist-test-slave-77v9 krb5kdc[5707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:21 dist-test-slave-77v9 krb5kdc[5707](info): set up 2 sockets
Aug 11 22:20:21 dist-test-slave-77v9 krb5kdc[5707](info): commencing operation
krb5kdc: starting...
W20260811 22:20:22.283656 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:22 dist-test-slave-77v9 krb5kdc[5707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:22.424096 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46495
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:22.549036 5722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:22.549431 5722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:22.549522 5722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:22.553809 5722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:22.553913 5722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:22.553947 5722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:22.553987 5722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:22.554049 5722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:22.559808 5722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:46495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:22.561101 5722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:22.562222 5722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:22.571483 5729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:22.571476 5731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:22.571503 5728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:22.571894 5722 server_base.cc:1034] running on GCE node
I20260811 22:20:22.572505 5722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:22.573825 5722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:22.575047 5722 hybrid_clock.cc:648] HybridClock initialized: now 1786486822575024 us; error 36 us; skew 500 ppm
Aug 11 22:20:22 dist-test-slave-77v9 krb5kdc[5707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486822, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:22.583699 5722 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:22.585172 5722 webserver.cc:460] Webserver started at http://127.17.198.62:38549/ using document root <none> and password file <none>
I20260811 22:20:22.585837 5722 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:22.585913 5722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:22.586095 5722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:22.588445 5722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "f50265e9ec004299970f8157cb15c15a"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "050fb32e199fd04584f7a5a059c30958"
server_key_iv: "e699b7838dbf3495b14ff7dd92e5e62d"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.589113 5722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "f50265e9ec004299970f8157cb15c15a"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "050fb32e199fd04584f7a5a059c30958"
server_key_iv: "e699b7838dbf3495b14ff7dd92e5e62d"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.593330 5722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:20:22.596012 5739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:22.597563 5722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:22.597741 5722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "f50265e9ec004299970f8157cb15c15a"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "050fb32e199fd04584f7a5a059c30958"
server_key_iv: "e699b7838dbf3495b14ff7dd92e5e62d"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.597880 5722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:22.617010 5722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:22.620347 5722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:22.620597 5722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:22.628674 5722 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:46495
I20260811 22:20:22.628684 5793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:46495 every 8 connection(s)
I20260811 22:20:22.629750 5722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:22.632695 5794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:22.632768 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5722
I20260811 22:20:22.632908 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:22.633246 18200 external_mini_cluster.cc:1428] Setting key 2f25990433b5fa6faedd8f8a73e92372
I20260811 22:20:22.640007 5794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: Bootstrap starting.
I20260811 22:20:22.642969 5794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: Neither blocks nor log segments found. Creating new log.
Aug 11 22:20:22 dist-test-slave-77v9 krb5kdc[5707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:22.643760 5794 log.cc:824] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:22.646122 5794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: No bootstrap required, opened a new log
I20260811 22:20:22.649111 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:22.634796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.17.198.62:46495)
0811 22:20:22.635250 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:22.635267 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:22.635305 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:20:22.636097 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:22.636116 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:22.636474 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:22.636779 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:22.637848 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.638938 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.640186 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.640678 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.640887 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:22.644015 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:22.644019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:22.644032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:22.644060 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:22.645936 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:22.646469 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:22.646489 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:22.646494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:22.646605 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:22.646912 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:22.646915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:22.646917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:22.647335 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:22.647548 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:22.647863 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 22:20:22.648227 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":291,"thread_start_us":147,"threads_started":1}
I20260811 22:20:22.650511 5794 raft_consensus.cc:348] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } }
I20260811 22:20:22.650813 5794 raft_consensus.cc:374] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:22.650872 5794 raft_consensus.cc:729] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50265e9ec004299970f8157cb15c15a, State: Initialized, Role: FOLLOWER
I20260811 22:20:22.651387 5794 consensus_queue.cc:260] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } }
I20260811 22:20:22.651562 5794 raft_consensus.cc:388] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:22.651638 5794 raft_consensus.cc:482] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:22.651715 5794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:22.652695 5794 raft_consensus.cc:504] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } }
I20260811 22:20:22.653080 5794 leader_election.cc:302] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f50265e9ec004299970f8157cb15c15a; no voters:
I20260811 22:20:22.653376 5794 leader_election.cc:288] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:22.653535 5799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:22.653811 5799 raft_consensus.cc:686] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [term 1 LEADER]: Becoming Leader. State: Replica: f50265e9ec004299970f8157cb15c15a, State: Running, Role: LEADER
I20260811 22:20:22.654160 5799 consensus_queue.cc:237] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } }
I20260811 22:20:22.654744 5794 sys_catalog.cc:564] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:22.655967 5800 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f50265e9ec004299970f8157cb15c15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } } }
I20260811 22:20:22.656111 5800 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:22.656423 5801 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f50265e9ec004299970f8157cb15c15a. Latest consensus state: current_term: 1 leader_uuid: "f50265e9ec004299970f8157cb15c15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50265e9ec004299970f8157cb15c15a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 46495 } } }
I20260811 22:20:22.656531 5801 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:22.656500 5810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:22.660054 5810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:22.665763 5810 catalog_manager.cc:1347] Generated new cluster ID: 6b2110ba15f44ac68eb0faca896011fb
I20260811 22:20:22.665851 5810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:22.684803 5810 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:22.686390 5810 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:22.694327 5810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: Generated new TSK 0
I20260811 22:20:22.695189 5810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:20:22.741465 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:22.879703 5822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:22.879995 5822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:22.880043 5822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:22.884421 5822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:22.884510 5822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:22.884573 5822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:22.890832 5822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:22.892015 5822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:22.893082 5822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:22.903586 5828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:22.903620 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:22.903635 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:22.903895 5822 server_base.cc:1034] running on GCE node
I20260811 22:20:22.904402 5822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:22.905138 5822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:22.906333 5822 hybrid_clock.cc:648] HybridClock initialized: now 1786486822906322 us; error 34 us; skew 500 ppm
Aug 11 22:20:22 dist-test-slave-77v9 krb5kdc[5707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486822, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:22.909479 5822 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:20:22.910584 5822 webserver.cc:460] Webserver started at http://127.17.198.1:34619/ using document root <none> and password file <none>
I20260811 22:20:22.911232 5822 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:22.911329 5822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:22.911581 5822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:22.914006 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "75292397794940cf93290084f184a01c"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "5c90679bdf46717c9af57b85da12f836"
server_key_iv: "7046029d3508bc54e7804112b3737768"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.914642 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "75292397794940cf93290084f184a01c"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "5c90679bdf46717c9af57b85da12f836"
server_key_iv: "7046029d3508bc54e7804112b3737768"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.919164 5822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:22.922047 5839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:22.923383 5822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:22.923540 5822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "75292397794940cf93290084f184a01c"
format_stamp: "Formatted at 2026-08-11 22:20:22 on dist-test-slave-77v9"
server_key: "5c90679bdf46717c9af57b85da12f836"
server_key_iv: "7046029d3508bc54e7804112b3737768"
server_key_version: "encryptionkey@0"
I20260811 22:20:22.923681 5822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:22.966858 5822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:22.970324 5822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:22.970512 5822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:22.971094 5822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:22.972007 5822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:22.972061 5822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:22.972105 5822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:22.972121 5822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:22.981640 5822 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:36083
I20260811 22:20:22.981662 5966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:36083 every 8 connection(s)
I20260811 22:20:22.982724 5822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:22.991348 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5822
I20260811 22:20:22.991520 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:22.991959 18200 external_mini_cluster.cc:1428] Setting key 76ba4db1f56c5b56b0df51aff038d21c
Aug 11 22:20:22 dist-test-slave-77v9 krb5kdc[5707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486822, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:22.997562 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:22.984713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:41471 (local address 127.17.198.62:46495)
0811 22:20:22.984903 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:22.984907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:22.985771 (+ 864us) server_negotiation.cc:407] Connection header received
0811 22:20:22.987334 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:22.987338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:22.987399 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:22.987500 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:22.988731 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.989342 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.990529 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.990765 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.990875 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:22.993947 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:22.993950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:22.993953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:22.993977 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:22.995426 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:22.995955 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:22.995959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:22.995961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:22.996043 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:22.996398 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:22.996401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:22.996403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:22.996587 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:22.996693 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:22.997260 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 22:20:22.997399 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:20:22.998440 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:22.985139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46495 (local address 127.17.198.1:41471)
0811 22:20:22.985612 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:22.985673 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:22.987111 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:22.987676 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:22.987683 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:22.988073 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:22.988568 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:22.988582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.989485 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.989488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:22.990388 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:22.990398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:22.990925 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:22.990930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:22.991169 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:22.991904 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:22.991933 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:22.993738 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:22.995551 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:22.995557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:22.995573 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:22.995846 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:22.996138 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:22.996141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:22.996143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:22.996293 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:22.996694 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:22.996704 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:22.997022 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 22:20:22.997219 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 22:20:22.997575 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
I20260811 22:20:22.999889 5967 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46495
I20260811 22:20:23.000183 5967 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:23.000697 5967 heartbeater.cc:511] Master 127.17.198.62:46495 requested a full tablet report, sending...
I20260811 22:20:23.002318 5756 ts_manager.cc:194] Registered new tserver with Master: 75292397794940cf93290084f184a01c (127.17.198.1:36083)
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:20:23.003554 5756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:41471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:20:23.025342 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:23.146937 5973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:23.147267 5973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:23.147315 5973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:23.151701 5973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:23.151805 5973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:23.151914 5973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:23.157990 5973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:23.159219 5973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:23.160348 5973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:23.171141 5978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:23.171164 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:23.171190 5979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:23.171250 5973 server_base.cc:1034] running on GCE node
I20260811 22:20:23.171991 5973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:23.172880 5973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:23.174096 5973 hybrid_clock.cc:648] HybridClock initialized: now 1786486823174059 us; error 51 us; skew 500 ppm
Aug 11 22:20:23 dist-test-slave-77v9 krb5kdc[5707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486823, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:23.177770 5973 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:20:23.179005 5973 webserver.cc:460] Webserver started at http://127.17.198.2:46303/ using document root <none> and password file <none>
I20260811 22:20:23.179651 5973 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:23.179719 5973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:23.179955 5973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:23.182271 5973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "9613f1d024684bf48701a190717b1d63"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "5b86b5c16e40f44d7ed10a41934eed0e"
server_key_iv: "c2b84a12b4c3ac2d8c5713b35bc5a85a"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.182960 5973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "9613f1d024684bf48701a190717b1d63"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "5b86b5c16e40f44d7ed10a41934eed0e"
server_key_iv: "c2b84a12b4c3ac2d8c5713b35bc5a85a"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.187165 5973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:23.189893 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.191417 5973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:23.191577 5973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "9613f1d024684bf48701a190717b1d63"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "5b86b5c16e40f44d7ed10a41934eed0e"
server_key_iv: "c2b84a12b4c3ac2d8c5713b35bc5a85a"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.191715 5973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:23.230000 5973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:23.233814 5973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:23.234052 5973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:23.234730 5973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:23.235630 5973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:23.235683 5973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.235724 5973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:23.235739 5973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.246124 5973 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:38565
I20260811 22:20:23.246177 6111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:38565 every 8 connection(s)
I20260811 22:20:23.247193 5973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:23.254343 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 5973
I20260811 22:20:23.254532 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:23.254842 18200 external_mini_cluster.cc:1428] Setting key 71ac9feb446ade6754fb206bb964c724
Aug 11 22:20:23 dist-test-slave-77v9 krb5kdc[5707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486823, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:23.263273 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.249303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:34927 (local address 127.17.198.62:46495)
0811 22:20:23.249473 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.249476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.250354 (+ 878us) server_negotiation.cc:407] Connection header received
0811 22:20:23.251899 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.251903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.251969 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.252080 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:23.253398 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.253960 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.255348 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.255578 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.255694 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.258725 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:23.258732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:23.258738 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:23.258777 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:23.260431 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.261214 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.261218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.261219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.261313 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.261797 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.261800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.261802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.262067 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:23.262150 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.262985 (+ 835us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.263134 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:20:23.264634 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.249672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46495 (local address 127.17.198.2:34927)
0811 22:20:23.250183 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.250229 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.251654 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.252274 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.252283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.252695 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:23.253237 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.253251 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.254101 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.254105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.255223 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.255233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.255742 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.255746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.255933 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.256596 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:23.256614 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:23.258489 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:23.260578 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:23.260588 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:23.260611 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:23.261096 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:23.261438 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:23.261444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:23.261449 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:23.261678 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:23.262182 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:23.262192 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:23.262627 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.262891 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.263350 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":143,"threads_started":1}
I20260811 22:20:23.266170 6112 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46495
I20260811 22:20:23.266467 6112 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
I20260811 22:20:23.267132 6112 heartbeater.cc:511] Master 127.17.198.62:46495 requested a full tablet report, sending...
I20260811 22:20:23.268401 5756 ts_manager.cc:194] Registered new tserver with Master: 9613f1d024684bf48701a190717b1d63 (127.17.198.2:38565)
I20260811 22:20:23.269157 5756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:34927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:20:23.288326 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:23.410499 6117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:23.410796 6117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:23.410842 6117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:23.414927 6117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:23.415009 6117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:23.415069 6117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:23.420495 6117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:46495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:23.421674 6117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:23.422737 6117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:23.434633 6125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:23.434635 6127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:23.434788 6117 server_base.cc:1034] running on GCE node
W20260811 22:20:23.434681 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:23.435386 6117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:23.436039 6117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:23.437256 6117 hybrid_clock.cc:648] HybridClock initialized: now 1786486823437226 us; error 45 us; skew 500 ppm
Aug 11 22:20:23 dist-test-slave-77v9 krb5kdc[5707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486823, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:23.440178 6117 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:20:23.441259 6117 webserver.cc:460] Webserver started at http://127.17.198.3:37051/ using document root <none> and password file <none>
I20260811 22:20:23.441869 6117 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:23.441967 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:23.442202 6117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:23.444509 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "3985560ad6d64f8882376b2d3a428a61"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "673706d8056c61182eeb365de94133ef"
server_key_iv: "8c5b5891d325caac9eaef65bdebf081d"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.445158 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "3985560ad6d64f8882376b2d3a428a61"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "673706d8056c61182eeb365de94133ef"
server_key_iv: "8c5b5891d325caac9eaef65bdebf081d"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.449522 6117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:23.452343 6134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.454123 6117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:23.454326 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "3985560ad6d64f8882376b2d3a428a61"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "673706d8056c61182eeb365de94133ef"
server_key_iv: "8c5b5891d325caac9eaef65bdebf081d"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.454483 6117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:23.469923 6117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:23.473423 6117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:23.473675 6117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:23.474323 6117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:23.475195 6117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:23.475273 6117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.475348 6117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:23.475397 6117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.485662 6117 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:42325
I20260811 22:20:23.485684 6262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:42325 every 8 connection(s)
I20260811 22:20:23.486989 6117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:23.487659 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6117
I20260811 22:20:23.487826 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:23.488194 18200 external_mini_cluster.cc:1428] Setting key 4d1d2cf22f464b3204c11c77c36b19c5
Aug 11 22:20:23 dist-test-slave-77v9 krb5kdc[5707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486823, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:23.501998 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.489115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:60029 (local address 127.17.198.62:46495)
0811 22:20:23.489274 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.489278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.490359 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 22:20:23.491846 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.491848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.491898 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.491981 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:23.493264 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.493808 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.494988 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.495199 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.495287 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.498103 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:23.498106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:23.498108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:23.498133 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:23.499537 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.500206 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.500209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.500210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.500289 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.500624 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.500626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.500627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.500876 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:23.501014 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.501642 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.501796 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:20:23.502944 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.489685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:46495 (local address 127.17.198.3:60029)
0811 22:20:23.490183 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.490237 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.491630 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.492138 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.492144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.492550 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:23.493119 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.493132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.493924 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.493927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.494867 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.494877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.495354 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.495357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.495501 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.496266 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:23.496283 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:23.497917 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:23.499652 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:23.499658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:23.499671 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:23.500095 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:23.500370 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:23.500373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:23.500374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:23.500538 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:23.501014 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:23.501022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:23.501320 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.501574 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.501965 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":135,"threads_started":1}
I20260811 22:20:23.504045 6263 heartbeater.cc:347] Connected to a master server at 127.17.198.62:46495
I20260811 22:20:23.504287 6263 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:23.504791 6263 heartbeater.cc:511] Master 127.17.198.62:46495 requested a full tablet report, sending...
I20260811 22:20:23.505892 5756 ts_manager.cc:194] Registered new tserver with Master: 3985560ad6d64f8882376b2d3a428a61 (127.17.198.3:42325)
I20260811 22:20:23.506500 5756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:60029
I20260811 22:20:23.515089 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:23.526610 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.517390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.17.198.62:46495)
0811 22:20:23.517640 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.517644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.517768 (+ 124us) server_negotiation.cc:407] Connection header received
0811 22:20:23.517898 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.517900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.517957 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.518022 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:23.518857 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.519421 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.520317 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.520573 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.520677 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.522387 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:23.522391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:23.522393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:23.522420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:23.524241 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.525014 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.525017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.525019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.525128 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:23.525469 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:23.525472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:23.525473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:23.525738 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:23.525951 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.526237 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.526399 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260811 22:20:23.530628 5756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:23.534034 5756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:23.549860 6274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.543461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:42325 (local address 127.0.0.1:39496)
0811 22:20:23.543895 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.543961 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.544095 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.545322 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.545328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.545350 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:23.545512 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.545519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.546644 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.546647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.547965 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.547977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.549191 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.549194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.549351 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.549465 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.549539 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.549694 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":71,"thread_start_us":106,"threads_started":1}
I20260811 22:20:23.549893 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.543648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.17.198.3:42325)
0811 22:20:23.544782 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.544796 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.544848 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:20:23.544929 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.544935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.545162 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.545336 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:23.545665 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.546491 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.548113 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.549037 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.549435 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.549580 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.549637 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.549733 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":177,"threads_started":1}
I20260811 22:20:23.549894 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.544489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.17.198.2:38565)
0811 22:20:23.544910 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.544969 (+ 59us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.545019 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:20:23.545177 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.545182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.545381 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.545528 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:23.545752 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.546951 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.548357 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.549020 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.549140 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.549298 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.549666 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.549754 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":135,"threads_started":1}
I20260811 22:20:23.550521 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.543497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36083 (local address 127.0.0.1:57118)
0811 22:20:23.544400 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.544427 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.544528 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.545743 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.545747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.545767 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:23.545862 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.545868 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.547565 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.547568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.548894 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.548906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.550101 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.550107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.550199 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.550223 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.550276 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.550377 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":88,"thread_start_us":96,"threads_started":1}
I20260811 22:20:23.550848 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.544294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.17.198.1:36083)
0811 22:20:23.545297 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.545303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.545322 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:20:23.545391 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.545397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.545582 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.546020 (+ 438us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:23.546163 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.547437 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.549054 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.549950 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.550386 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.550527 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.550586 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.550697 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"thread_start_us":74,"threads_started":1}
I20260811 22:20:23.550928 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.544224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:38565 (local address 127.0.0.1:60672)
0811 22:20:23.544937 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.544951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.545043 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.545519 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.545522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.545539 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:23.545661 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.545667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.547096 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.547099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.548236 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.548243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.549163 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.549170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.549358 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.549430 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.549547 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.550797 (+ 1250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":807,"thread_start_us":93,"threads_started":1}
I20260811 22:20:23.553336 5894 tablet_service.cc:1467] Processing CreateTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b (DEFAULT_TABLE table=test-table [id=02b5a11c0ebf4a628b68db499b714cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:23.553351 6191 tablet_service.cc:1467] Processing CreateTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b (DEFAULT_TABLE table=test-table [id=02b5a11c0ebf4a628b68db499b714cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:23.553949 6042 tablet_service.cc:1467] Processing CreateTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b (DEFAULT_TABLE table=test-table [id=02b5a11c0ebf4a628b68db499b714cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:23.554382 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969f8caafd14e49ad23fe422fedcd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:23.554447 6191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969f8caafd14e49ad23fe422fedcd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:23.554956 6042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969f8caafd14e49ad23fe422fedcd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:23.561450 6281 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: Bootstrap starting.
I20260811 22:20:23.562791 6282 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Bootstrap starting.
I20260811 22:20:23.563740 6281 tablet_bootstrap.cc:654] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:23.564531 6281 log.cc:824] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:23.564642 6282 tablet_bootstrap.cc:654] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:23.565165 6283 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Bootstrap starting.
I20260811 22:20:23.566102 6282 log.cc:824] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:23.566800 6281 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: No bootstrap required, opened a new log
I20260811 22:20:23.567022 6281 ts_tablet_manager.cc:1397] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:20:23.568197 6283 tablet_bootstrap.cc:654] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:23.568421 6282 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: No bootstrap required, opened a new log
I20260811 22:20:23.568646 6282 ts_tablet_manager.cc:1397] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:20:23.569175 6283 log.cc:824] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:23.571396 6281 raft_consensus.cc:348] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.571661 6283 tablet_bootstrap.cc:492] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: No bootstrap required, opened a new log
I20260811 22:20:23.571671 6281 raft_consensus.cc:374] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:23.571735 6281 raft_consensus.cc:729] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75292397794940cf93290084f184a01c, State: Initialized, Role: FOLLOWER
I20260811 22:20:23.571892 6283 ts_tablet_manager.cc:1397] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:20:23.572258 6281 consensus_queue.cc:260] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.572583 6282 raft_consensus.cc:348] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.572912 6282 raft_consensus.cc:374] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:23.572989 6282 raft_consensus.cc:729] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3985560ad6d64f8882376b2d3a428a61, State: Initialized, Role: FOLLOWER
I20260811 22:20:23.573030 5967 heartbeater.cc:503] Master 127.17.198.62:46495 was elected leader, sending a full tablet report...
I20260811 22:20:23.573443 6282 consensus_queue.cc:260] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.574632 6282 ts_tablet_manager.cc:1428] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:20:23.575037 6263 heartbeater.cc:503] Master 127.17.198.62:46495 was elected leader, sending a full tablet report...
I20260811 22:20:23.576102 6283 raft_consensus.cc:348] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.576458 6283 raft_consensus.cc:374] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:23.576548 6283 raft_consensus.cc:729] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9613f1d024684bf48701a190717b1d63, State: Initialized, Role: FOLLOWER
I20260811 22:20:23.576777 6281 ts_tablet_manager.cc:1428] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260811 22:20:23.577073 6283 consensus_queue.cc:260] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.578240 6283 ts_tablet_manager.cc:1428] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:20:23.578634 6112 heartbeater.cc:503] Master 127.17.198.62:46495 was elected leader, sending a full tablet report...
I20260811 22:20:23.647667 6289 raft_consensus.cc:482] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:23.647876 6289 raft_consensus.cc:504] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.649092 6289 leader_election.cc:288] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3985560ad6d64f8882376b2d3a428a61 (127.17.198.3:42325), 75292397794940cf93290084f184a01c (127.17.198.1:36083)
I20260811 22:20:23.653247 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.649515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:58431 (local address 127.17.198.3:42325)
0811 22:20:23.649732 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.649737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.650004 (+ 267us) server_negotiation.cc:407] Connection header received
0811 22:20:23.650161 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.650164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.650213 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.650298 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:23.650572 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.651072 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652200 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.652618 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652757 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.652814 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.653042 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.653119 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 22:20:23.653263 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.649408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:42325 (local address 127.17.198.2:58431)
0811 22:20:23.649895 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.649907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.650044 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.650310 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.650313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.650329 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:23.650458 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.650464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.651190 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.651195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.652048 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.652058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.652757 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.652904 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.652928 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.653025 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.653101 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":130,"thread_start_us":131,"threads_started":1}
I20260811 22:20:23.653386 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.649395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36083 (local address 127.17.198.2:39245)
0811 22:20:23.649524 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.649545 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:23.649676 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:23.650037 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:23.650039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:23.650060 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:23.650191 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.650197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.651013 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.651018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.652045 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:23.652056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.652865 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:23.652997 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.653015 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:20:23.653106 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:20:23.653188 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":58}
I20260811 22:20:23.653366 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.649555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:39245 (local address 127.17.198.1:36083)
0811 22:20:23.649696 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.649702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.649735 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:20:23.649851 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.649854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.649904 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.650023 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:23.650316 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.650886 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652202 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.652693 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.652869 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.652929 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.653138 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.653226 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:20:23.654037 6211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969f8caafd14e49ad23fe422fedcd4b" candidate_uuid: "9613f1d024684bf48701a190717b1d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3985560ad6d64f8882376b2d3a428a61" is_pre_election: true
I20260811 22:20:23.654107 5914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969f8caafd14e49ad23fe422fedcd4b" candidate_uuid: "9613f1d024684bf48701a190717b1d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75292397794940cf93290084f184a01c" is_pre_election: true
I20260811 22:20:23.654418 6211 raft_consensus.cc:2454] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9613f1d024684bf48701a190717b1d63 in term 0.
I20260811 22:20:23.654476 5914 raft_consensus.cc:2454] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9613f1d024684bf48701a190717b1d63 in term 0.
I20260811 22:20:23.655014 5992 leader_election.cc:302] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3985560ad6d64f8882376b2d3a428a61, 75292397794940cf93290084f184a01c, 9613f1d024684bf48701a190717b1d63; no voters:
I20260811 22:20:23.655354 6289 raft_consensus.cc:2781] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:23.655486 6289 raft_consensus.cc:482] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:23.655550 6289 raft_consensus.cc:3037] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:23.656914 6289 raft_consensus.cc:504] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.657343 6289 leader_election.cc:288] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [CANDIDATE]: Term 1 election: Requested vote from peers 3985560ad6d64f8882376b2d3a428a61 (127.17.198.3:42325), 75292397794940cf93290084f184a01c (127.17.198.1:36083)
I20260811 22:20:23.657814 5914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969f8caafd14e49ad23fe422fedcd4b" candidate_uuid: "9613f1d024684bf48701a190717b1d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75292397794940cf93290084f184a01c"
I20260811 22:20:23.657979 5914 raft_consensus.cc:3037] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:23.658097 6211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969f8caafd14e49ad23fe422fedcd4b" candidate_uuid: "9613f1d024684bf48701a190717b1d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3985560ad6d64f8882376b2d3a428a61"
I20260811 22:20:23.658306 6211 raft_consensus.cc:3037] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:23.659485 5914 raft_consensus.cc:2454] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9613f1d024684bf48701a190717b1d63 in term 1.
I20260811 22:20:23.659647 6211 raft_consensus.cc:2454] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9613f1d024684bf48701a190717b1d63 in term 1.
I20260811 22:20:23.659926 5993 leader_election.cc:302] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75292397794940cf93290084f184a01c, 9613f1d024684bf48701a190717b1d63; no voters:
I20260811 22:20:23.660148 6289 raft_consensus.cc:2781] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:23.660450 6289 raft_consensus.cc:686] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 1 LEADER]: Becoming Leader. State: Replica: 9613f1d024684bf48701a190717b1d63, State: Running, Role: LEADER
I20260811 22:20:23.660877 6289 consensus_queue.cc:237] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } }
I20260811 22:20:23.664891 5755 catalog_manager.cc:5495] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9613f1d024684bf48701a190717b1d63 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "9613f1d024684bf48701a190717b1d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9613f1d024684bf48701a190717b1d63" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 38565 } health_report { overall_health: HEALTHY } } }
I20260811 22:20:23.677803 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.673953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60680 (local address 127.17.198.2:38565)
0811 22:20:23.674114 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.674117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.674133 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:23.674337 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.674340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.674415 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.674525 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:23.674830 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.675353 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.676191 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.676422 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.676508 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.676775 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:23.677306 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:23.677422 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.677553 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.677594 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:20:23.684274 6211 raft_consensus.cc:1261] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 1 FOLLOWER]: Refusing update from remote peer 9613f1d024684bf48701a190717b1d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:23.684298 5914 raft_consensus.cc:1261] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 1 FOLLOWER]: Refusing update from remote peer 9613f1d024684bf48701a190717b1d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:23.685072 6289 consensus_queue.cc:1035] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3985560ad6d64f8882376b2d3a428a61" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 42325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:23.685292 6291 consensus_queue.cc:1035] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75292397794940cf93290084f184a01c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 36083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:23.697225 6295 mvcc.cc:205] Tried to move back new op lower bound from 7317450029802364928 to 7317450029717860352. Current Snapshot: MvccSnapshot[applied={T|T < 7317450029802364928 or (T in {7317450029802364928})}]
I20260811 22:20:23.702906 5756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:23.703112 5756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:23.705679 5756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f50265e9ec004299970f8157cb15c15a: Sending DeleteTablet for 3 replicas of tablet 6969f8caafd14e49ad23fe422fedcd4b
I20260811 22:20:23.706660 6042 tablet_service.cc:1514] Processing DeleteTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:23 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:60672
I20260811 22:20:23.706665 5894 tablet_service.cc:1514] Processing DeleteTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:23 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:57118
I20260811 22:20:23.706794 6191 tablet_service.cc:1514] Processing DeleteTablet for tablet 6969f8caafd14e49ad23fe422fedcd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:23 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:39496
I20260811 22:20:23.707458 6305 tablet_replica.cc:330] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c: stopping tablet replica
I20260811 22:20:23.707472 6307 tablet_replica.cc:330] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: stopping tablet replica
I20260811 22:20:23.707829 6305 raft_consensus.cc:2229] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:23.707829 6307 raft_consensus.cc:2229] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:23.708004 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5822
I20260811 22:20:23.708235 6307 raft_consensus.cc:2258] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:23.708388 6305 raft_consensus.cc:2258] T 6969f8caafd14e49ad23fe422fedcd4b P 75292397794940cf93290084f184a01c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:23.708925 6308 tablet_replica.cc:330] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: stopping tablet replica
I20260811 22:20:23.709316 6308 raft_consensus.cc:2229] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:23.709717 6308 raft_consensus.cc:2258] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:23.711194 6307 ts_tablet_manager.cc:1905] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:23.711766 6308 ts_tablet_manager.cc:1905] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:23.714583 6307 ts_tablet_manager.cc:1918] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:23.714723 6307 log.cc:1196] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/6969f8caafd14e49ad23fe422fedcd4b
I20260811 22:20:23.715138 6307 ts_tablet_manager.cc:1939] T 6969f8caafd14e49ad23fe422fedcd4b P 9613f1d024684bf48701a190717b1d63: Deleting consensus metadata
I20260811 22:20:23.715273 6308 ts_tablet_manager.cc:1918] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:23.715461 6308 log.cc:1196] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/6969f8caafd14e49ad23fe422fedcd4b
I20260811 22:20:23.715898 6308 ts_tablet_manager.cc:1939] T 6969f8caafd14e49ad23fe422fedcd4b P 3985560ad6d64f8882376b2d3a428a61: Deleting consensus metadata
I20260811 22:20:23.717023 5742 catalog_manager.cc:4841] TS 9613f1d024684bf48701a190717b1d63 (127.17.198.2:38565): tablet 6969f8caafd14e49ad23fe422fedcd4b (table test-table [id=02b5a11c0ebf4a628b68db499b714cc0]) successfully deleted
I20260811 22:20:23.717300 5742 catalog_manager.cc:4841] TS 3985560ad6d64f8882376b2d3a428a61 (127.17.198.3:42325): tablet 6969f8caafd14e49ad23fe422fedcd4b (table test-table [id=02b5a11c0ebf4a628b68db499b714cc0]) successfully deleted
W20260811 22:20:23.722322 5743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:36083 (error 108)
I20260811 22:20:23.723101 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5973
I20260811 22:20:23.724171 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.723327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:36083 (local address 127.0.0.1:57134)
0811 22:20:23.723861 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:23.724035 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420}
W20260811 22:20:23.727020 5743 catalog_manager.cc:4584] TS 75292397794940cf93290084f184a01c (127.17.198.1:36083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6969f8caafd14e49ad23fe422fedcd4b: Network error: Client connection negotiation failed: client connection to 127.17.198.1:36083: connect: Connection refused (error 111)
I20260811 22:20:23.732553 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6117
I20260811 22:20:23.742617 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 5722
2026-08-11T22:20:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2582 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:20:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:23Z Disabled control of system clock
I20260811 22:20:23.780547 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:44905
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:44905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:23.900118 6313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:23.900501 6313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:23.900578 6313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:23.904918 6313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:23.905026 6313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:23.905056 6313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:23.905099 6313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:23.905135 6313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:23.910271 6313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:44905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:44905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:23.911468 6313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:23.912964 6313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:23.921715 6318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:23.921715 6320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:23.921718 6322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:23.922129 6313 server_base.cc:1034] running on GCE node
I20260811 22:20:23.922763 6313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:23.924010 6313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:23.925222 6313 hybrid_clock.cc:648] HybridClock initialized: now 1786486823925200 us; error 35 us; skew 500 ppm
I20260811 22:20:23.927412 6313 webserver.cc:460] Webserver started at http://127.17.198.62:40519/ using document root <none> and password file <none>
I20260811 22:20:23.928016 6313 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:23.928114 6313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:23.928349 6313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:23.930603 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "2c54cd7be0af4e4c8bce5953aaff82f0"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "1a5ae645ab795e8a01714192e4d31a9a"
server_key_iv: "9edb0ead7f2a99c2543db324af4cbfcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.931217 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "2c54cd7be0af4e4c8bce5953aaff82f0"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "1a5ae645ab795e8a01714192e4d31a9a"
server_key_iv: "9edb0ead7f2a99c2543db324af4cbfcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.935236 6313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 22:20:23.937831 6329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:23.939057 6313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:20:23.939208 6313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "2c54cd7be0af4e4c8bce5953aaff82f0"
format_stamp: "Formatted at 2026-08-11 22:20:23 on dist-test-slave-77v9"
server_key: "1a5ae645ab795e8a01714192e4d31a9a"
server_key_iv: "9edb0ead7f2a99c2543db324af4cbfcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:23.939332 6313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:23.954926 6313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:23.955724 6313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:23.955952 6313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:23.963517 6313 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:44905
I20260811 22:20:23.963523 6391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:44905 every 8 connection(s)
I20260811 22:20:23.964581 6313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:23.967972 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6313
I20260811 22:20:23.968137 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:23.968151 6392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:23.968433 18200 external_mini_cluster.cc:1428] Setting key 3070cc6f815374a02b5b6bb8cef930b0
I20260811 22:20:23.975116 6392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: Bootstrap starting.
I20260811 22:20:23.977869 6392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:23.978351 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:23.969938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.17.198.62:44905)
0811 22:20:23.970711 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:23.970735 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:23.970784 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:20:23.971516 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:23.971533 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:23.971943 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:23.972295 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:23.972745 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.973803 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.974768 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:23.975263 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:23.975461 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:23.976386 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:23.976391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:23.976405 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:23.976882 (+ 477us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:23.976953 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:23.976977 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:23.977063 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:23.977222 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:20:23.977546 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":157,"threads_started":1}
I20260811 22:20:23.978771 6392 log.cc:824] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:23.981205 6392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: No bootstrap required, opened a new log
I20260811 22:20:23.984980 6392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } }
I20260811 22:20:23.985239 6392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:23.985288 6392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c54cd7be0af4e4c8bce5953aaff82f0, State: Initialized, Role: FOLLOWER
I20260811 22:20:23.985721 6392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } }
I20260811 22:20:23.985855 6392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:23.985904 6392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:23.986001 6392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:23.987284 6392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } }
I20260811 22:20:23.987661 6392 leader_election.cc:302] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c54cd7be0af4e4c8bce5953aaff82f0; no voters:
I20260811 22:20:23.988008 6392 leader_election.cc:288] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:23.988165 6397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:23.988416 6397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [term 1 LEADER]: Becoming Leader. State: Replica: 2c54cd7be0af4e4c8bce5953aaff82f0, State: Running, Role: LEADER
I20260811 22:20:23.988754 6397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } }
I20260811 22:20:23.989349 6392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:23.990479 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c54cd7be0af4e4c8bce5953aaff82f0. Latest consensus state: current_term: 1 leader_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } } }
I20260811 22:20:23.990622 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:23.991034 6409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:23.991472 6398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c54cd7be0af4e4c8bce5953aaff82f0" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44905 } } }
I20260811 22:20:23.991575 6398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:23.994443 6409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:24.000481 6409 catalog_manager.cc:1347] Generated new cluster ID: fdfd7593186c45ffa757ed9194ebe78c
I20260811 22:20:24.000574 6409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:24.017899 6409 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:24.019068 6409 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:24.024905 6409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: Generated new TSK 0
I20260811 22:20:24.025672 6409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:24.032042 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--builtin_ntp_servers=127.17.198.20:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:24.153936 6421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:24.154256 6421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:24.154300 6421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:24.158669 6421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:24.158766 6421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:24.158879 6421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:24.164038 6421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:24.165343 6421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:24.166884 6421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:24.177712 6427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:24.177700 6426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:24.178079 6421 server_base.cc:1034] running on GCE node
W20260811 22:20:24.177700 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:24.178751 6421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:24.179610 6421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:24.180871 6421 hybrid_clock.cc:648] HybridClock initialized: now 1786486824180799 us; error 32 us; skew 500 ppm
I20260811 22:20:24.182961 6421 webserver.cc:460] Webserver started at http://127.17.198.1:36053/ using document root <none> and password file <none>
I20260811 22:20:24.183578 6421 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:24.183673 6421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:24.183912 6421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:24.186183 6421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "abab65f343454204b90ad1c200947d2e"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "f625aa8c74593539fe11705d48cb9767"
server_key_iv: "2995e656b681f2422312e39b4a2483de"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.186812 6421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "abab65f343454204b90ad1c200947d2e"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "f625aa8c74593539fe11705d48cb9767"
server_key_iv: "2995e656b681f2422312e39b4a2483de"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.190999 6421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:24.193819 6436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.195101 6421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:20:24.195252 6421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "abab65f343454204b90ad1c200947d2e"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "f625aa8c74593539fe11705d48cb9767"
server_key_iv: "2995e656b681f2422312e39b4a2483de"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.195398 6421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:24.207736 6421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:24.208483 6421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:24.208698 6421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:24.209375 6421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:24.210459 6421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:24.210531 6421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.210598 6421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:24.210647 6421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.219888 6421 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:32993
I20260811 22:20:24.219901 6560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:32993 every 8 connection(s)
I20260811 22:20:24.220891 6421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:24.230283 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6421
I20260811 22:20:24.230321 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.222746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:59445 (local address 127.17.198.62:44905)
0811 22:20:24.222969 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.222973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.224013 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 22:20:24.225546 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.225549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.225601 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.225683 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:24.226622 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.227173 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.228279 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.228498 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.228578 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.229736 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:24.229739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:24.229741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:24.229754 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:24.229775 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:24.229783 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:24.229843 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.230068 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.230146 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:20:24.230453 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:24.230720 18200 external_mini_cluster.cc:1428] Setting key dc0f80a65e731f13d43b5a7762e1bd4d
I20260811 22:20:24.231251 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.223163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44905 (local address 127.17.198.1:59445)
0811 22:20:24.223807 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.223901 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.225324 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.225878 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.225885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.225970 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:24.226466 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.226489 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.227289 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.227291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.228175 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.228183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.228632 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.228635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.228777 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.229486 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:24.229488 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:24.229528 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:24.229590 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:24.229901 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:24.229909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:24.229913 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.230028 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.230255 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260811 22:20:24.232499 6561 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44905
I20260811 22:20:24.232738 6561 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:24.232895 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--builtin_ntp_servers=127.17.198.20:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:24.233322 6561 heartbeater.cc:511] Master 127.17.198.62:44905 requested a full tablet report, sending...
I20260811 22:20:24.235023 6349 ts_manager.cc:194] Registered new tserver with Master: abab65f343454204b90ad1c200947d2e (127.17.198.1:32993)
I20260811 22:20:24.236231 6349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.1:59445
W20260811 22:20:24.354374 6564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:24.354707 6564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:24.354755 6564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:24.359063 6564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:24.359153 6564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:24.359216 6564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:24.364389 6564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:24.365614 6564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:24.367092 6564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:24.377990 6564 server_base.cc:1034] running on GCE node
W20260811 22:20:24.377938 6571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:24.377991 6575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:24.377938 6572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:24.378764 6564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:24.379521 6564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:24.380744 6564 hybrid_clock.cc:648] HybridClock initialized: now 1786486824380712 us; error 52 us; skew 500 ppm
I20260811 22:20:24.383001 6564 webserver.cc:460] Webserver started at http://127.17.198.2:34685/ using document root <none> and password file <none>
I20260811 22:20:24.383692 6564 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:24.383797 6564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:24.384053 6564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:24.386582 6564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "c0dcc6c384444942967f5c17402d32ce"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "80700c7d52927692c746bf39d63934f8"
server_key_iv: "0450000c83048d80b92597382477ce38"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.387235 6564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "c0dcc6c384444942967f5c17402d32ce"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "80700c7d52927692c746bf39d63934f8"
server_key_iv: "0450000c83048d80b92597382477ce38"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.391855 6564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:20:24.394819 6581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.396271 6564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:20:24.396442 6564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "c0dcc6c384444942967f5c17402d32ce"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "80700c7d52927692c746bf39d63934f8"
server_key_iv: "0450000c83048d80b92597382477ce38"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.396585 6564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:24.430405 6564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:24.431243 6564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:24.431468 6564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:24.432106 6564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:24.433202 6564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:24.433284 6564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.433368 6564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:24.433409 6564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.443015 6564 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:45215
I20260811 22:20:24.443041 6704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:45215 every 8 connection(s)
I20260811 22:20:24.444016 6564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:24.451556 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6564
I20260811 22:20:24.451683 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:24.451928 18200 external_mini_cluster.cc:1428] Setting key aa5a265778b85cb8ed6c9513fc131ed2
I20260811 22:20:24.454540 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.446094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:48909 (local address 127.17.198.62:44905)
0811 22:20:24.446259 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.446264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.447530 (+ 1266us) server_negotiation.cc:407] Connection header received
0811 22:20:24.449050 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.449053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.449107 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.449194 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:24.450187 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.450746 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.452100 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.452417 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.452518 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.453970 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:24.453974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:24.453977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:24.453994 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:24.454020 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:24.454031 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:24.454108 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.454271 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.454353 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:20:24.454675 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--builtin_ntp_servers=127.17.198.20:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:24.455508 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.446618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44905 (local address 127.17.198.2:48909)
0811 22:20:24.447361 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.447407 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.448747 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.449380 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.449387 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.449479 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:24.450037 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.450052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.450865 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.450868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.451923 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.451935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.452615 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.452620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.452807 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.453608 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:24.453611 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:24.453675 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:24.453776 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:24.454128 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:24.454136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:24.454141 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.454243 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.454528 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":128,"threads_started":1}
I20260811 22:20:24.456902 6705 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44905
I20260811 22:20:24.457168 6705 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:24.457783 6705 heartbeater.cc:511] Master 127.17.198.62:44905 requested a full tablet report, sending...
I20260811 22:20:24.459062 6349 ts_manager.cc:194] Registered new tserver with Master: c0dcc6c384444942967f5c17402d32ce (127.17.198.2:45215)
I20260811 22:20:24.459687 6349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.2:48909
W20260811 22:20:24.574889 6708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:24.575174 6708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:24.575214 6708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:24.579552 6708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:24.579634 6708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:24.579694 6708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:24.585016 6708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:24.586314 6708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:24.587805 6708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:24.598702 6716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:24.598852 6708 server_base.cc:1034] running on GCE node
W20260811 22:20:24.598718 6719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:24.598753 6717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:24.599534 6708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:24.600348 6708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:24.601572 6708 hybrid_clock.cc:648] HybridClock initialized: now 1786486824601552 us; error 36 us; skew 500 ppm
I20260811 22:20:24.603705 6708 webserver.cc:460] Webserver started at http://127.17.198.3:34569/ using document root <none> and password file <none>
I20260811 22:20:24.604346 6708 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:24.604450 6708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:24.604667 6708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:24.606938 6708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "167a2ab14c5a4b588a26b36cce96b1da"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "b091ae8181d8f7829c91614b8dc016ae"
server_key_iv: "042497f90a3c27c6352a6ffabeae54cd"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.607609 6708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "167a2ab14c5a4b588a26b36cce96b1da"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "b091ae8181d8f7829c91614b8dc016ae"
server_key_iv: "042497f90a3c27c6352a6ffabeae54cd"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.611640 6708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:20:24.614243 6725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.615545 6708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:20:24.615686 6708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "167a2ab14c5a4b588a26b36cce96b1da"
format_stamp: "Formatted at 2026-08-11 22:20:24 on dist-test-slave-77v9"
server_key: "b091ae8181d8f7829c91614b8dc016ae"
server_key_iv: "042497f90a3c27c6352a6ffabeae54cd"
server_key_version: "encryptionkey@0"
I20260811 22:20:24.615805 6708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:24.638695 6708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:24.639524 6708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:24.639756 6708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:24.640424 6708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:24.641518 6708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:24.641594 6708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.641667 6708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:24.641708 6708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:24.656909 6708 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:34865
I20260811 22:20:24.656946 6847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:34865 every 8 connection(s)
I20260811 22:20:24.657914 6708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:24.663388 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6708
I20260811 22:20:24.663544 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:24.663972 18200 external_mini_cluster.cc:1428] Setting key 9abb84ababf2dda8b6bb4b61a7ea3c84
I20260811 22:20:24.668100 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.659956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:38765 (local address 127.17.198.62:44905)
0811 22:20:24.660132 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.660137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.661233 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 22:20:24.662623 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.662626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.662682 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.662782 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:24.663777 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.664502 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.666048 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.666391 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.666520 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.667655 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:24.667659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:24.667661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:24.667674 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:24.667696 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:24.667704 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:24.667771 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.667911 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.667981 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:20:24.669138 6850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.660355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44905 (local address 127.17.198.3:38765)
0811 22:20:24.661083 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.661126 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.662393 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.662953 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.662960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.663044 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:24.663580 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.663607 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.664635 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.664639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.665918 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.665931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.666576 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.666580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.666730 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.667368 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:24.667370 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:24.667414 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:24.667508 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:24.667783 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:24.667790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:24.667794 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.667897 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.668148 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":121,"threads_started":1}
I20260811 22:20:24.670320 6848 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44905
I20260811 22:20:24.670567 6848 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:24.671142 6848 heartbeater.cc:511] Master 127.17.198.62:44905 requested a full tablet report, sending...
I20260811 22:20:24.672262 6349 ts_manager.cc:194] Registered new tserver with Master: 167a2ab14c5a4b588a26b36cce96b1da (127.17.198.3:34865)
I20260811 22:20:24.673161 6349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.3:38765
I20260811 22:20:24.678346 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:24.684237 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.680335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.17.198.62:44905)
0811 22:20:24.680483 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.680488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.680619 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:20:24.680723 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.680725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.680788 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.680903 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:24.681172 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.681713 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.682577 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.682831 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.682914 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.683709 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:24.683712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:24.683714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:24.683727 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:24.683747 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:24.683755 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:24.683864 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.684008 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.684068 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:20:24.687634 6349 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:24.691073 6349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:24.706655 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.701011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.17.198.3:34865)
0811 22:20:24.701564 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.701575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.701596 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:20:24.701667 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.701672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.701849 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.701985 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:24.702311 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703532 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.704955 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.705789 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.705934 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.706157 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.706328 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.706441 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":196,"threads_started":1}
I20260811 22:20:24.706655 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.699889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:45215 (local address 127.0.0.1:49188)
0811 22:20:24.700391 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.700441 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.700583 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.701480 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.701486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.701514 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:24.701682 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.701689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703649 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.703652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.704965 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.704976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.705970 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.705974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.706112 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.706164 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.706269 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.706447 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":54,"thread_start_us":130,"threads_started":1}
I20260811 22:20:24.706665 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.699988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:34865 (local address 127.0.0.1:52664)
0811 22:20:24.701237 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.701252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.701354 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.702091 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.702094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.702109 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:24.702211 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.702216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703799 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.703805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.704779 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.704788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.705941 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.705945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.706108 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.706161 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.706284 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.706459 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":132,"thread_start_us":125,"threads_started":1}
I20260811 22:20:24.707341 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.700643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:32993 (local address 127.0.0.1:45198)
0811 22:20:24.701824 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.701835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.701957 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.702477 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.702480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.702498 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:24.702586 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.702592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703540 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.703547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.705577 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.705586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.706914 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.706930 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.707038 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.707061 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.707137 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.707205 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":101,"threads_started":1}
I20260811 22:20:24.707839 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.700845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.17.198.1:32993)
0811 22:20:24.701697 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.701706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.701932 (+ 226us) server_negotiation.cc:407] Connection header received
0811 22:20:24.702093 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.702098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.702351 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.702501 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:24.702681 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703423 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.705704 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.706532 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.706637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.706840 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.707599 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.707710 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":103,"threads_started":1}
I20260811 22:20:24.707855 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.700023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.17.198.2:45215)
0811 22:20:24.700923 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.700939 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.700964 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:20:24.701046 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.701052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.701282 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.701836 (+ 554us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:24.702008 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.703532 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.705104 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.705829 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.707341 (+ 1512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.707542 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.707597 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.707691 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":119,"threads_started":1}
I20260811 22:20:24.710254 6632 tablet_service.cc:1467] Processing CreateTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 (DEFAULT_TABLE table=test-table [id=d13baf92f9634652905632b35fe1ff91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:24.710306 6776 tablet_service.cc:1467] Processing CreateTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 (DEFAULT_TABLE table=test-table [id=d13baf92f9634652905632b35fe1ff91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:24.711249 6632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292d2c9f2e144d82add97eddb01c6f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:24.710836 6488 tablet_service.cc:1467] Processing CreateTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 (DEFAULT_TABLE table=test-table [id=d13baf92f9634652905632b35fe1ff91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:24.711386 6776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292d2c9f2e144d82add97eddb01c6f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:24.711788 6488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292d2c9f2e144d82add97eddb01c6f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:24.718572 6866 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Bootstrap starting.
I20260811 22:20:24.719733 6870 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Bootstrap starting.
I20260811 22:20:24.720029 6868 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: Bootstrap starting.
I20260811 22:20:24.721511 6866 tablet_bootstrap.cc:654] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:24.722061 6870 tablet_bootstrap.cc:654] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:24.722091 6868 tablet_bootstrap.cc:654] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:24.722433 6866 log.cc:824] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:24.722936 6870 log.cc:824] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:24.722942 6868 log.cc:824] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:24.725013 6868 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: No bootstrap required, opened a new log
I20260811 22:20:24.725191 6866 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: No bootstrap required, opened a new log
I20260811 22:20:24.725234 6868 ts_tablet_manager.cc:1397] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 22:20:24.725286 6870 tablet_bootstrap.cc:492] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: No bootstrap required, opened a new log
I20260811 22:20:24.725400 6866 ts_tablet_manager.cc:1397] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:20:24.725454 6870 ts_tablet_manager.cc:1397] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:20:24.728600 6866 raft_consensus.cc:348] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.728953 6866 raft_consensus.cc:374] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:24.729014 6866 raft_consensus.cc:729] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0dcc6c384444942967f5c17402d32ce, State: Initialized, Role: FOLLOWER
I20260811 22:20:24.729487 6866 consensus_queue.cc:260] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.729768 6870 raft_consensus.cc:348] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.729768 6868 raft_consensus.cc:348] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.730048 6870 raft_consensus.cc:374] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:24.730048 6868 raft_consensus.cc:374] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:24.730111 6870 raft_consensus.cc:729] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167a2ab14c5a4b588a26b36cce96b1da, State: Initialized, Role: FOLLOWER
I20260811 22:20:24.730110 6868 raft_consensus.cc:729] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abab65f343454204b90ad1c200947d2e, State: Initialized, Role: FOLLOWER
I20260811 22:20:24.730400 6705 heartbeater.cc:503] Master 127.17.198.62:44905 was elected leader, sending a full tablet report...
I20260811 22:20:24.730641 6866 ts_tablet_manager.cc:1428] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:20:24.730626 6870 consensus_queue.cc:260] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.730626 6868 consensus_queue.cc:260] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.731426 6848 heartbeater.cc:503] Master 127.17.198.62:44905 was elected leader, sending a full tablet report...
I20260811 22:20:24.731699 6868 ts_tablet_manager.cc:1428] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:20:24.731699 6870 ts_tablet_manager.cc:1428] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:20:24.732071 6561 heartbeater.cc:503] Master 127.17.198.62:44905 was elected leader, sending a full tablet report...
W20260811 22:20:24.909063 6849 tablet.cc:2369] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:24.917590 6876 raft_consensus.cc:482] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:24.917845 6876 raft_consensus.cc:504] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.919198 6876 leader_election.cc:288] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0dcc6c384444942967f5c17402d32ce (127.17.198.2:45215), 167a2ab14c5a4b588a26b36cce96b1da (127.17.198.3:34865)
I20260811 22:20:24.923594 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.919639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:59219 (local address 127.17.198.2:45215)
0811 22:20:24.919790 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.919795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.920093 (+ 298us) server_negotiation.cc:407] Connection header received
0811 22:20:24.920270 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.920273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.920321 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.920409 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:24.920757 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.921330 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.922561 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.922949 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.923079 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.923136 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.923366 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.923456 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:20:24.923594 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.919646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:57697 (local address 127.17.198.3:34865)
0811 22:20:24.919803 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.919808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.920167 (+ 359us) server_negotiation.cc:407] Connection header received
0811 22:20:24.920327 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.920335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.920389 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.920470 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:24.920777 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.921339 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.922555 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.922959 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.923087 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.923139 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.923372 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.923449 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:20:24.923615 6878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.919508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:34865 (local address 127.17.198.1:57697)
0811 22:20:24.920064 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.920077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.920212 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.920480 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.920483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.920515 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:24.920670 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.920675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.921453 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.921458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.922433 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.922442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.923080 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.923084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.923207 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.923270 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.923355 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.923447 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":148,"thread_start_us":130,"threads_started":1}
I20260811 22:20:24.923619 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.919508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:45215 (local address 127.17.198.1:59219)
0811 22:20:24.919966 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.919979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:24.920150 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:24.920422 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:24.920425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:24.920442 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:24.920639 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.920649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.921465 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.921470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.922435 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:24.922445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.923078 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.923082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:24.923243 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.923262 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:20:24.923348 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:20:24.923456 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":164,"thread_start_us":150,"threads_started":1}
I20260811 22:20:24.924413 6800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292d2c9f2e144d82add97eddb01c6f01" candidate_uuid: "abab65f343454204b90ad1c200947d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167a2ab14c5a4b588a26b36cce96b1da" is_pre_election: true
I20260811 22:20:24.924410 6654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292d2c9f2e144d82add97eddb01c6f01" candidate_uuid: "abab65f343454204b90ad1c200947d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dcc6c384444942967f5c17402d32ce" is_pre_election: true
I20260811 22:20:24.924742 6654 raft_consensus.cc:2454] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abab65f343454204b90ad1c200947d2e in term 0.
I20260811 22:20:24.924737 6800 raft_consensus.cc:2454] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abab65f343454204b90ad1c200947d2e in term 0.
I20260811 22:20:24.925278 6438 leader_election.cc:302] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 167a2ab14c5a4b588a26b36cce96b1da, abab65f343454204b90ad1c200947d2e; no voters:
I20260811 22:20:24.925573 6876 raft_consensus.cc:2781] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:24.925654 6876 raft_consensus.cc:482] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:24.925704 6876 raft_consensus.cc:3037] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:24.926838 6876 raft_consensus.cc:504] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.927222 6876 leader_election.cc:288] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [CANDIDATE]: Term 1 election: Requested vote from peers c0dcc6c384444942967f5c17402d32ce (127.17.198.2:45215), 167a2ab14c5a4b588a26b36cce96b1da (127.17.198.3:34865)
I20260811 22:20:24.927641 6800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292d2c9f2e144d82add97eddb01c6f01" candidate_uuid: "abab65f343454204b90ad1c200947d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167a2ab14c5a4b588a26b36cce96b1da"
I20260811 22:20:24.927641 6654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292d2c9f2e144d82add97eddb01c6f01" candidate_uuid: "abab65f343454204b90ad1c200947d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dcc6c384444942967f5c17402d32ce"
I20260811 22:20:24.927786 6800 raft_consensus.cc:3037] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:24.927786 6654 raft_consensus.cc:3037] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:24.929221 6654 raft_consensus.cc:2454] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abab65f343454204b90ad1c200947d2e in term 1.
I20260811 22:20:24.929221 6800 raft_consensus.cc:2454] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abab65f343454204b90ad1c200947d2e in term 1.
I20260811 22:20:24.929611 6439 leader_election.cc:302] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 167a2ab14c5a4b588a26b36cce96b1da, abab65f343454204b90ad1c200947d2e, c0dcc6c384444942967f5c17402d32ce; no voters:
I20260811 22:20:24.929811 6876 raft_consensus.cc:2781] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:24.930109 6876 raft_consensus.cc:686] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [term 1 LEADER]: Becoming Leader. State: Replica: abab65f343454204b90ad1c200947d2e, State: Running, Role: LEADER
I20260811 22:20:24.930505 6876 consensus_queue.cc:237] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } }
I20260811 22:20:24.933887 6349 catalog_manager.cc:5495] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e reported cstate change: term changed from 0 to 1, leader changed from <none> to abab65f343454204b90ad1c200947d2e (127.17.198.1). New cstate: current_term: 1 leader_uuid: "abab65f343454204b90ad1c200947d2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abab65f343454204b90ad1c200947d2e" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 32993 } health_report { overall_health: HEALTHY } } }
W20260811 22:20:24.945227 6706 tablet.cc:2369] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:24.948988 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.945060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.17.198.1:32993)
0811 22:20:24.945214 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:24.945219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:24.945235 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:24.945427 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:24.945432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:24.945509 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:24.945624 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:24.945880 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.946395 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.947301 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:24.947533 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:24.947645 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:24.947991 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:24.948474 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:24.948649 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:24.948761 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:20:24.948832 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:20:24.955458 6800 raft_consensus.cc:1261] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 1 FOLLOWER]: Refusing update from remote peer abab65f343454204b90ad1c200947d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:24.955457 6654 raft_consensus.cc:1261] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 1 FOLLOWER]: Refusing update from remote peer abab65f343454204b90ad1c200947d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:24.956406 6876 consensus_queue.cc:1035] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0dcc6c384444942967f5c17402d32ce" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 45215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:24.956645 6880 consensus_queue.cc:1035] T 292d2c9f2e144d82add97eddb01c6f01 P abab65f343454204b90ad1c200947d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "167a2ab14c5a4b588a26b36cce96b1da" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:24.964524 6884 mvcc.cc:205] Tried to move back new op lower bound from 7317450035009351680 to 7317450034918223872. Current Snapshot: MvccSnapshot[applied={T|T < 7317450035009351680}]
I20260811 22:20:24.964975 6885 mvcc.cc:205] Tried to move back new op lower bound from 7317450035009351680 to 7317450034918223872. Current Snapshot: MvccSnapshot[applied={T|T < 7317450035009351680}]
I20260811 22:20:24.967495 6883 mvcc.cc:205] Tried to move back new op lower bound from 7317450035009351680 to 7317450034918223872. Current Snapshot: MvccSnapshot[applied={T|T < 7317450035009351680}]
I20260811 22:20:24.978468 6349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:24.978667 6349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:24.981384 6349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c54cd7be0af4e4c8bce5953aaff82f0: Sending DeleteTablet for 3 replicas of tablet 292d2c9f2e144d82add97eddb01c6f01
I20260811 22:20:24.982321 6776 tablet_service.cc:1514] Processing DeleteTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:24 UTC) from {username='slave'} at 127.0.0.1:52664
I20260811 22:20:24.982313 6488 tablet_service.cc:1514] Processing DeleteTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:24 UTC) from {username='slave'} at 127.0.0.1:45198
I20260811 22:20:24.982288 6632 tablet_service.cc:1514] Processing DeleteTablet for tablet 292d2c9f2e144d82add97eddb01c6f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:24 UTC) from {username='slave'} at 127.0.0.1:49188
I20260811 22:20:24.983148 6893 tablet_replica.cc:330] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: stopping tablet replica
I20260811 22:20:24.983525 6893 raft_consensus.cc:2229] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:24.983791 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6421
I20260811 22:20:24.983901 6893 raft_consensus.cc:2258] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:24.984548 6894 tablet_replica.cc:330] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: stopping tablet replica
I20260811 22:20:24.984982 6894 raft_consensus.cc:2229] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:24.985337 6894 raft_consensus.cc:2258] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:24.986596 6893 ts_tablet_manager.cc:1905] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:24.987145 6894 ts_tablet_manager.cc:1905] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:24.989702 6893 ts_tablet_manager.cc:1918] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:24.989831 6893 log.cc:1196] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/292d2c9f2e144d82add97eddb01c6f01
I20260811 22:20:24.990206 6893 ts_tablet_manager.cc:1939] T 292d2c9f2e144d82add97eddb01c6f01 P c0dcc6c384444942967f5c17402d32ce: Deleting consensus metadata
I20260811 22:20:24.990175 6894 ts_tablet_manager.cc:1918] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:24.990289 6894 log.cc:1196] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/292d2c9f2e144d82add97eddb01c6f01
I20260811 22:20:24.990691 6894 ts_tablet_manager.cc:1939] T 292d2c9f2e144d82add97eddb01c6f01 P 167a2ab14c5a4b588a26b36cce96b1da: Deleting consensus metadata
I20260811 22:20:24.991737 6334 catalog_manager.cc:4841] TS c0dcc6c384444942967f5c17402d32ce (127.17.198.2:45215): tablet 292d2c9f2e144d82add97eddb01c6f01 (table test-table [id=d13baf92f9634652905632b35fe1ff91]) successfully deleted
I20260811 22:20:24.991858 6333 catalog_manager.cc:4841] TS 167a2ab14c5a4b588a26b36cce96b1da (127.17.198.3:34865): tablet 292d2c9f2e144d82add97eddb01c6f01 (table test-table [id=d13baf92f9634652905632b35fe1ff91]) successfully deleted
W20260811 22:20:24.994668 6333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:32993 (error 108)
I20260811 22:20:24.994943 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6564
I20260811 22:20:24.998080 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:24.997722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:32993 (local address 127.0.0.1:45206)
0811 22:20:24.997903 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:24.997996 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:20:24.998410 6333 catalog_manager.cc:4584] TS abab65f343454204b90ad1c200947d2e (127.17.198.1:32993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 292d2c9f2e144d82add97eddb01c6f01: Network error: Client connection negotiation failed: client connection to 127.17.198.1:32993: connect: Connection refused (error 111)
I20260811 22:20:25.002990 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6708
I20260811 22:20:25.010032 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6313
2026-08-11T22:20:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1264 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:25 dist-test-slave-77v9 krb5kdc[6898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:25 dist-test-slave-77v9 krb5kdc[6898](info): set up 2 sockets
Aug 11 22:20:25 dist-test-slave-77v9 krb5kdc[6898](info): commencing operation
krb5kdc: starting...
W20260811 22:20:26.155869 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:26 dist-test-slave-77v9 krb5kdc[6898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:26.305959 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42711
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:26.427418 6911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:26.427724 6911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:26.427767 6911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:26.432154 6911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:26.432246 6911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:26.432267 6911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:26.432284 6911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:26.432300 6911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:26.438391 6911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:26.439674 6911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:26.440831 6911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:26.450922 6919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:26.451193 6911 server_base.cc:1034] running on GCE node
W20260811 22:20:26.450959 6917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:26.450979 6916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:26.452183 6911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:26.453679 6911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:26.454886 6911 hybrid_clock.cc:648] HybridClock initialized: now 1786486826454865 us; error 42 us; skew 500 ppm
Aug 11 22:20:26 dist-test-slave-77v9 krb5kdc[6898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486826, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:26.458334 6911 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:26.459396 6911 webserver.cc:460] Webserver started at http://127.17.198.62:36825/ using document root <none> and password file <none>
I20260811 22:20:26.460060 6911 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:26.460167 6911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:26.460430 6911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:26.462764 6911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "29aee08f8269411286199f8bcbc36489"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "d707cd47d7f5ecf777ccb9aaaf35aa56"
server_key_iv: "9be10661fe409e1943b7287a0182b0d2"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.463397 6911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "29aee08f8269411286199f8bcbc36489"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "d707cd47d7f5ecf777ccb9aaaf35aa56"
server_key_iv: "9be10661fe409e1943b7287a0182b0d2"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.467679 6911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:20:26.470499 6926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:26.472116 6911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:20:26.472287 6911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "29aee08f8269411286199f8bcbc36489"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "d707cd47d7f5ecf777ccb9aaaf35aa56"
server_key_iv: "9be10661fe409e1943b7287a0182b0d2"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.472460 6911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:26.489573 6911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:26.493162 6911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:26.493412 6911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:26.501381 6911 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:42711
I20260811 22:20:26.501410 6984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:42711 every 8 connection(s)
I20260811 22:20:26.502485 6911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:26.504468 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 6911
I20260811 22:20:26.504655 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:26.505038 18200 external_mini_cluster.cc:1428] Setting key fd2de76dfddfc6dd5de69380851f807c
I20260811 22:20:26.506188 6985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:26.512667 6985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: Bootstrap starting.
Aug 11 22:20:26 dist-test-slave-77v9 krb5kdc[6898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:26.515771 6985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:26.516705 6985 log.cc:824] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:26.519019 6985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: No bootstrap required, opened a new log
I20260811 22:20:26.520517 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:26.506651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.17.198.62:42711)
0811 22:20:26.507190 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:26.507207 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:26.507247 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:20:26.508009 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:26.508028 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:26.508379 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:26.508757 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:26.509718 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.510759 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.511670 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.512188 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.512376 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:26.515328 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:26.515341 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:26.515378 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:26.515422 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:26.517324 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:26.517863 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:26.517869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:26.517874 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:26.518005 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:26.518317 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:26.518321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:26.518322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:26.518781 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:26.519015 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:26.519331 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:20:26.519675 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":197,"threads_started":1}
I20260811 22:20:26.523577 6985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } }
I20260811 22:20:26.523882 6985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:26.523967 6985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29aee08f8269411286199f8bcbc36489, State: Initialized, Role: FOLLOWER
I20260811 22:20:26.524456 6985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } }
I20260811 22:20:26.524638 6985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:26.524727 6985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:26.524879 6985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:26.526072 6985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } }
I20260811 22:20:26.526453 6985 leader_election.cc:302] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29aee08f8269411286199f8bcbc36489; no voters:
I20260811 22:20:26.526906 6985 leader_election.cc:288] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:26.527101 6990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:26.527369 6990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [term 1 LEADER]: Becoming Leader. State: Replica: 29aee08f8269411286199f8bcbc36489, State: Running, Role: LEADER
I20260811 22:20:26.527750 6990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } }
I20260811 22:20:26.528334 6985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:26.529633 6992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29aee08f8269411286199f8bcbc36489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } } }
I20260811 22:20:26.529780 6992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:26.530120 6991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29aee08f8269411286199f8bcbc36489. Latest consensus state: current_term: 1 leader_uuid: "29aee08f8269411286199f8bcbc36489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aee08f8269411286199f8bcbc36489" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42711 } } }
I20260811 22:20:26.530243 6991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489 [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:26.534147 7005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:26.534251 7005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:26.534363 6999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:26.535202 6999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:26.541771 6999 catalog_manager.cc:1347] Generated new cluster ID: 1e1c2189800f44178e73438fb1a56a42
I20260811 22:20:26.541878 6999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:26.551453 6999 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:26.552551 6999 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:26.563110 6999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: Generated new TSK 0
I20260811 22:20:26.564000 6999 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.198.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
Entry for principal kudu/127.17.198.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab.
I20260811 22:20:26.601779 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:26.725560 7012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:26.725919 7012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:26.725972 7012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:26.731240 7012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:26.731336 7012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:26.731438 7012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:26.737967 7012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.1.keytab
--principal=kudu/127.17.198.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:26.739305 7012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:26.740554 7012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:26.752252 7022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:26.752264 7017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:26.752420 7012 server_base.cc:1034] running on GCE node
W20260811 22:20:26.752283 7019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:26.753067 7012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:26.753800 7012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:26.755007 7012 hybrid_clock.cc:648] HybridClock initialized: now 1786486826754969 us; error 51 us; skew 500 ppm
Aug 11 22:20:26 dist-test-slave-77v9 krb5kdc[6898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486826, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:26.758271 7012 init.cc:377] Logged in from keytab as kudu/127.17.198.1@KRBTEST.COM (short username kudu)
I20260811 22:20:26.759208 7012 webserver.cc:460] Webserver started at http://127.17.198.1:34893/ using document root <none> and password file <none>
I20260811 22:20:26.759846 7012 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:26.759917 7012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:26.760093 7012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:26.762398 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "37a481dbc37f4c7b9272ead656acedda"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "fb21938b3631a01c3c9270bf7a74e272"
server_key_iv: "3f4b4e3e3bf459878e051ce7f5e17b43"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.762989 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "37a481dbc37f4c7b9272ead656acedda"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "fb21938b3631a01c3c9270bf7a74e272"
server_key_iv: "3f4b4e3e3bf459878e051ce7f5e17b43"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.767405 7012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:26.770512 7029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:26.772054 7012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:20:26.772190 7012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "37a481dbc37f4c7b9272ead656acedda"
format_stamp: "Formatted at 2026-08-11 22:20:26 on dist-test-slave-77v9"
server_key: "fb21938b3631a01c3c9270bf7a74e272"
server_key_iv: "3f4b4e3e3bf459878e051ce7f5e17b43"
server_key_version: "encryptionkey@0"
I20260811 22:20:26.772307 7012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:26.799222 7012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:26.802835 7012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:26.803010 7012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:26.803620 7012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:26.804454 7012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:26.804502 7012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:26.804541 7012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:26.804557 7012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:26.813988 7012 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:35237
I20260811 22:20:26.814020 7155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:35237 every 8 connection(s)
I20260811 22:20:26.814930 7012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:26.821139 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7012
I20260811 22:20:26.821286 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:26.821609 18200 external_mini_cluster.cc:1428] Setting key d10bb9a11c1b8a3616b85a95505ec858
Aug 11 22:20:26 dist-test-slave-77v9 krb5kdc[6898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486826, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.1@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:26.830957 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:26.816866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:54635 (local address 127.17.198.62:42711)
0811 22:20:26.817026 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:26.817029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:26.817938 (+ 909us) server_negotiation.cc:407] Connection header received
0811 22:20:26.819322 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:26.819325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:26.819386 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:26.819469 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:26.820693 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.821673 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.823224 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.823461 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.823591 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:26.826546 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:26.826552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:26.826557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:26.826589 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:26.828445 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:26.829102 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:26.829106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:26.829109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:26.829215 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:26.829652 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:26.829656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:26.829657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:26.829863 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:26.830028 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:26.830608 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:20:26.830759 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:20:26.831840 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:26.817304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42711 (local address 127.17.198.1:54635)
0811 22:20:26.817772 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:26.817818 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:26.819086 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:26.819642 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:26.819648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:26.820032 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:26.820530 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:26.820545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.821808 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.821811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:26.823076 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:26.823087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:26.823633 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:26.823637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:26.823860 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:26.824585 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:26.824602 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:26.826331 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:26.828575 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:26.828581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:26.828596 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:26.829012 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:26.829356 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:26.829359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:26.829361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:26.829539 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:26.830024 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:26.830030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:26.830345 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 22:20:26.830550 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:20:26.830916 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.17.198.2@KRBTEST.COM; defaulting to no policy
I20260811 22:20:26.833284 7157 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42711
I20260811 22:20:26.833539 7157 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:26.834035 7157 heartbeater.cc:511] Master 127.17.198.62:42711 requested a full tablet report, sending...
I20260811 22:20:26.835747 6944 ts_manager.cc:194] Registered new tserver with Master: 37a481dbc37f4c7b9272ead656acedda (127.17.198.1:35237)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.2@KRBTEST.COM" created.
I20260811 22:20:26.837350 6944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.1@KRBTEST.COM'} at 127.17.198.1:54635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
Entry for principal kudu/127.17.198.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab.
I20260811 22:20:26.857383 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:26.980005 7162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:26.980319 7162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:26.980365 7162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:26.984784 7162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:26.984918 7162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:26.984990 7162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:26.990592 7162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.2.keytab
--principal=kudu/127.17.198.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:26.991792 7162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:26.992892 7162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:27.003001 7170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:27.003146 7167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:27.003196 7162 server_base.cc:1034] running on GCE node
W20260811 22:20:27.003257 7168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:27.003774 7162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:27.004443 7162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:27.005627 7162 hybrid_clock.cc:648] HybridClock initialized: now 1786486827005611 us; error 27 us; skew 500 ppm
Aug 11 22:20:27 dist-test-slave-77v9 krb5kdc[6898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486827, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:27.008857 7162 init.cc:377] Logged in from keytab as kudu/127.17.198.2@KRBTEST.COM (short username kudu)
I20260811 22:20:27.010105 7162 webserver.cc:460] Webserver started at http://127.17.198.2:44259/ using document root <none> and password file <none>
I20260811 22:20:27.010706 7162 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:27.010803 7162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:27.011020 7162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:27.013221 7162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "7fcb545ee2f04f90a60712789912850f"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "57c0bb4c190879de480cee0e966c020d"
server_key_iv: "d17c46e4b4679900fbe979c5077eea1d"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.013815 7162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "7fcb545ee2f04f90a60712789912850f"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "57c0bb4c190879de480cee0e966c020d"
server_key_iv: "d17c46e4b4679900fbe979c5077eea1d"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.017875 7162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:20:27.020459 7178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.021973 7162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:20:27.022120 7162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "7fcb545ee2f04f90a60712789912850f"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "57c0bb4c190879de480cee0e966c020d"
server_key_iv: "d17c46e4b4679900fbe979c5077eea1d"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.022244 7162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:27.067229 7162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:27.071300 7162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:27.071563 7162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:27.072302 7162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:27.073376 7162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:27.073443 7162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.073516 7162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:27.073560 7162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.084187 7162 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:33749
I20260811 22:20:27.084211 7299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:33749 every 8 connection(s)
I20260811 22:20:27.085290 7162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:27.086580 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7162
I20260811 22:20:27.086727 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:27.087119 18200 external_mini_cluster.cc:1428] Setting key 7dea9166332253f46226c424bc462827
Aug 11 22:20:27 dist-test-slave-77v9 krb5kdc[6898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486827, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.2@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.3@KRBTEST.COM" created.
I20260811 22:20:27.103271 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.087779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:37233 (local address 127.17.198.62:42711)
0811 22:20:27.087922 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.087927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.089183 (+ 1256us) server_negotiation.cc:407] Connection header received
0811 22:20:27.090556 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.090559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.090617 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.090721 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:27.092458 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.093162 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.095063 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.095313 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.095418 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.098890 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:27.098895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:27.098899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:27.098934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:27.100628 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.101283 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.101289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.101291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.101384 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.101858 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.101860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.101862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.102099 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:27.102220 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.102897 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.103082 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:20:27.104121 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.088168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42711 (local address 127.17.198.2:37233)
0811 22:20:27.089033 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.089078 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.090333 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.090895 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.090902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.091470 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:27.092256 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.092279 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.093306 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.093312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.094911 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.094928 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.095516 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.095523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.095799 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.096590 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:27.096616 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:27.098651 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:27.100777 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:27.100784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:27.100803 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:27.101167 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:27.101490 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:27.101494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:27.101519 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:27.101676 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:27.102231 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:27.102241 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:27.102614 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.102822 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.103204 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":452,"threads_started":1}
I20260811 22:20:27.105428 7300 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42711
I20260811 22:20:27.105683 7300 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:27.106230 7300 heartbeater.cc:511] Master 127.17.198.62:42711 requested a full tablet report, sending...
I20260811 22:20:27.107487 6944 ts_manager.cc:194] Registered new tserver with Master: 7fcb545ee2f04f90a60712789912850f (127.17.198.2:33749)
I20260811 22:20:27.108359 6944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.2@KRBTEST.COM'} at 127.17.198.2:37233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
Entry for principal kudu/127.17.198.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab.
I20260811 22:20:27.123692 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:27.240584 7305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:27.240901 7305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:27.240949 7305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:27.245105 7305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:27.245177 7305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:27.245265 7305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:27.250998 7305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.3.keytab
--principal=kudu/127.17.198.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:27.252141 7305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:27.253244 7305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:27.264192 7310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:27.264286 7305 server_base.cc:1034] running on GCE node
W20260811 22:20:27.264199 7311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:27.264199 7313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:27.265131 7305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:27.265826 7305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:27.267025 7305 hybrid_clock.cc:648] HybridClock initialized: now 1786486827266995 us; error 44 us; skew 500 ppm
Aug 11 22:20:27 dist-test-slave-77v9 krb5kdc[6898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486827, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:27.270222 7305 init.cc:377] Logged in from keytab as kudu/127.17.198.3@KRBTEST.COM (short username kudu)
I20260811 22:20:27.271283 7305 webserver.cc:460] Webserver started at http://127.17.198.3:34159/ using document root <none> and password file <none>
I20260811 22:20:27.271914 7305 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:27.271984 7305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:27.272214 7305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:27.274582 7305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "0cd0fea80f604c9098ccf443519a118b"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "01abf37fa47e51affd865325d1bec07e"
server_key_iv: "2132b32fa5ed656392b0d3929a9b91e3"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.275234 7305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "0cd0fea80f604c9098ccf443519a118b"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "01abf37fa47e51affd865325d1bec07e"
server_key_iv: "2132b32fa5ed656392b0d3929a9b91e3"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.279758 7305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:20:27.282572 7322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.283926 7305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:20:27.284173 7305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "0cd0fea80f604c9098ccf443519a118b"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "01abf37fa47e51affd865325d1bec07e"
server_key_iv: "2132b32fa5ed656392b0d3929a9b91e3"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.284303 7305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:27.321225 7305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:27.324924 7305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:27.325155 7305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:27.325842 7305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:27.326887 7305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:27.326963 7305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.327037 7305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:27.327087 7305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.337137 7305 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:40965
I20260811 22:20:27.337198 7443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:40965 every 8 connection(s)
I20260811 22:20:27.338172 7305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:27.342162 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7305
I20260811 22:20:27.342367 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:27.342705 18200 external_mini_cluster.cc:1428] Setting key 2b81d9558e547b85d7ac790ffb94ea54
Aug 11 22:20:27 dist-test-slave-77v9 krb5kdc[6898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486827, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.3@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:27.354324 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.340388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:37033 (local address 127.17.198.62:42711)
0811 22:20:27.340538 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.340542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.341511 (+ 969us) server_negotiation.cc:407] Connection header received
0811 22:20:27.343233 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.343237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.343302 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.343397 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:27.345054 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.345655 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.346971 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.347253 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.347379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.350349 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:27.350353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:27.350355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:27.350386 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:27.351910 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.352533 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.352536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.352538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.352621 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.353060 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.353062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.353064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.353275 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:27.353354 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.354051 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.354177 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:20:27.355257 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.340776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42711 (local address 127.17.198.3:37033)
0811 22:20:27.341352 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.341399 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.343004 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.343721 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.343728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.344129 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:27.344903 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.344922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.345808 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.345812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.346831 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.346842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.347425 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.347429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.347624 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.348353 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:27.348373 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:27.350130 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:27.352064 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:27.352070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:27.352085 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:27.352424 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:27.352706 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:27.352708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:27.352709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:27.352965 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:27.353422 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:27.353429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:27.353714 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.353946 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.354256 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260811 22:20:27.356745 7444 heartbeater.cc:347] Connected to a master server at 127.17.198.62:42711
I20260811 22:20:27.357064 7444 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:27.357658 7444 heartbeater.cc:511] Master 127.17.198.62:42711 requested a full tablet report, sending...
I20260811 22:20:27.358932 6943 ts_manager.cc:194] Registered new tserver with Master: 0cd0fea80f604c9098ccf443519a118b (127.17.198.3:40965)
I20260811 22:20:27.359577 6943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.198.3@KRBTEST.COM'} at 127.17.198.3:37033
I20260811 22:20:27.369475 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:27.380509 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.371616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.17.198.62:42711)
0811 22:20:27.371828 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.371833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.371977 (+ 144us) server_negotiation.cc:407] Connection header received
0811 22:20:27.372165 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.372168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.372226 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.372321 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:27.373183 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.373824 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.374748 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.375034 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.375153 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.376884 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:27.376892 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:27.376904 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:27.376936 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:27.378618 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.379102 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.379107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.379108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.379198 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:27.379500 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:27.379504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:27.379506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:27.379765 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:27.379922 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.380185 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.380315 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:20:27.383776 6943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:27.387178 6943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:27.404208 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.398368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.17.198.3:40965)
0811 22:20:27.398815 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.398829 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.398972 (+ 143us) server_negotiation.cc:407] Connection header received
0811 22:20:27.399107 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.399112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.399416 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.399581 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:27.399926 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.400731 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.402769 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.403396 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.403530 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.403707 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.403903 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.404023 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":122,"threads_started":1}
I20260811 22:20:27.404191 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.397080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:35237 (local address 127.0.0.1:37054)
0811 22:20:27.397879 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.397895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.397995 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.399521 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.399525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.399544 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:27.399669 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.399675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.401459 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.401465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.402439 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.402450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.403707 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.403715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.403833 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.403859 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.403916 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.404036 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":210,"thread_start_us":108,"threads_started":1}
I20260811 22:20:27.404786 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.397080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:33749 (local address 127.0.0.1:51434)
0811 22:20:27.397728 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.397793 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.397944 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.398877 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.398883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.398915 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:27.399134 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.399143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.400872 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.400880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.402607 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.402619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.404016 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.404020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.404121 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.404142 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.404540 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.404626 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":137,"spinlock_wait_cycles":896,"thread_start_us":94,"threads_started":1}
I20260811 22:20:27.405126 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.398422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.17.198.1:35237)
0811 22:20:27.398992 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.398999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.399024 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:20:27.399111 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.399123 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.399347 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.399846 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:27.399999 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.401312 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.402590 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.403537 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.404023 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.404180 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.404237 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.404755 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":117,"threads_started":1}
I20260811 22:20:27.405213 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.397585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.17.198.2:33749)
0811 22:20:27.398077 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.398090 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.398121 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:20:27.398186 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.398190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.398445 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.398628 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:27.399323 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.400661 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.402761 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.403669 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.403778 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.403949 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.404917 (+ 968us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.405048 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":125,"threads_started":1}
I20260811 22:20:27.404789 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.397930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:40965 (local address 127.0.0.1:54184)
0811 22:20:27.398856 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.398870 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.398984 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.399645 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.399648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.399666 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:27.399803 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.399810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.400961 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.400965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.402607 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.402619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.403534 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.403538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.403669 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.403711 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.403868 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.404638 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":72,"thread_start_us":122,"threads_started":1}
I20260811 22:20:27.408227 7227 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9862a150344dbbaafec39b591e5052 (DEFAULT_TABLE table=test-table [id=797ea639d35343b0a8bd4fe085d3d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:27.408380 7373 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9862a150344dbbaafec39b591e5052 (DEFAULT_TABLE table=test-table [id=797ea639d35343b0a8bd4fe085d3d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:27.408381 7082 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9862a150344dbbaafec39b591e5052 (DEFAULT_TABLE table=test-table [id=797ea639d35343b0a8bd4fe085d3d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:27.409603 7373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9862a150344dbbaafec39b591e5052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:27.409598 7082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9862a150344dbbaafec39b591e5052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:27.409605 7227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9862a150344dbbaafec39b591e5052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:27.416776 7461 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Bootstrap starting.
I20260811 22:20:27.418197 7463 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Bootstrap starting.
I20260811 22:20:27.418665 7462 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: Bootstrap starting.
I20260811 22:20:27.420236 7463 tablet_bootstrap.cc:654] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:27.421036 7461 tablet_bootstrap.cc:654] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:27.421069 7462 tablet_bootstrap.cc:654] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:27.421604 7463 log.cc:824] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:27.421973 7462 log.cc:824] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:27.421973 7461 log.cc:824] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:27.423857 7463 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: No bootstrap required, opened a new log
I20260811 22:20:27.424100 7463 ts_tablet_manager.cc:1397] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:20:27.424125 7461 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: No bootstrap required, opened a new log
I20260811 22:20:27.424141 7462 tablet_bootstrap.cc:492] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: No bootstrap required, opened a new log
I20260811 22:20:27.424321 7462 ts_tablet_manager.cc:1397] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:20:27.424326 7461 ts_tablet_manager.cc:1397] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 22:20:27.429085 7461 raft_consensus.cc:348] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.429560 7461 raft_consensus.cc:374] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:27.429625 7461 raft_consensus.cc:729] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fcb545ee2f04f90a60712789912850f, State: Initialized, Role: FOLLOWER
I20260811 22:20:27.429461 7462 raft_consensus.cc:348] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.429714 7462 raft_consensus.cc:374] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:27.429767 7462 raft_consensus.cc:729] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a481dbc37f4c7b9272ead656acedda, State: Initialized, Role: FOLLOWER
I20260811 22:20:27.430243 7462 consensus_queue.cc:260] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.430243 7461 consensus_queue.cc:260] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.431427 7461 ts_tablet_manager.cc:1428] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260811 22:20:27.431427 7462 ts_tablet_manager.cc:1428] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:20:27.431821 7300 heartbeater.cc:503] Master 127.17.198.62:42711 was elected leader, sending a full tablet report...
I20260811 22:20:27.431821 7157 heartbeater.cc:503] Master 127.17.198.62:42711 was elected leader, sending a full tablet report...
I20260811 22:20:27.438936 7463 raft_consensus.cc:348] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.439368 7463 raft_consensus.cc:374] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:27.439441 7463 raft_consensus.cc:729] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd0fea80f604c9098ccf443519a118b, State: Initialized, Role: FOLLOWER
I20260811 22:20:27.440059 7463 consensus_queue.cc:260] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.440964 7444 heartbeater.cc:503] Master 127.17.198.62:42711 was elected leader, sending a full tablet report...
I20260811 22:20:27.441534 7463 ts_tablet_manager.cc:1428] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260811 22:20:27.497145 7470 raft_consensus.cc:482] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:27.497359 7470 raft_consensus.cc:504] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.498519 7470 leader_election.cc:288] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37a481dbc37f4c7b9272ead656acedda (127.17.198.1:35237), 7fcb545ee2f04f90a60712789912850f (127.17.198.2:33749)
I20260811 22:20:27.503424 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.498799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:33749 (local address 127.17.198.3:49819)
0811 22:20:27.499007 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.499026 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.499129 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.499493 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.499496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.499514 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:27.499656 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.499661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.500709 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.500712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.502201 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.502213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.502948 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.502952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.503064 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.503100 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.503172 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.503252 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":49}
I20260811 22:20:27.503471 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.499001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:49819 (local address 127.17.198.2:33749)
0811 22:20:27.499124 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.499128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.499143 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:20:27.499306 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.499309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.499361 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.499491 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:27.499791 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.500580 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.502369 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.502814 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.502953 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.503008 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.503252 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.503336 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:20:27.503593 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.499050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:40163 (local address 127.17.198.1:35237)
0811 22:20:27.499191 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.499195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.499456 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:20:27.499595 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.499601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.499654 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.499716 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:27.500018 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.500730 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.502351 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.503002 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.503124 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.503175 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.503378 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.503479 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:20:27.503945 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.498799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:35237 (local address 127.17.198.3:40163)
0811 22:20:27.499346 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.499357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:27.499484 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:27.499770 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:27.499773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:27.499790 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:27.499898 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.499902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.500889 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.500893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.502201 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:27.502213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.503121 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.503127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:27.503225 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.503243 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:20:27.503735 (+ 492us) client_negotiation.cc:240] Negotiation successful
0811 22:20:27.503822 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":182,"thread_start_us":173,"threads_started":1}
I20260811 22:20:27.504311 7251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9862a150344dbbaafec39b591e5052" candidate_uuid: "0cd0fea80f604c9098ccf443519a118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fcb545ee2f04f90a60712789912850f" is_pre_election: true
I20260811 22:20:27.504664 7251 raft_consensus.cc:2454] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd0fea80f604c9098ccf443519a118b in term 0.
I20260811 22:20:27.505102 7104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9862a150344dbbaafec39b591e5052" candidate_uuid: "0cd0fea80f604c9098ccf443519a118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a481dbc37f4c7b9272ead656acedda" is_pre_election: true
I20260811 22:20:27.505244 7323 leader_election.cc:302] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd0fea80f604c9098ccf443519a118b, 7fcb545ee2f04f90a60712789912850f; no voters:
I20260811 22:20:27.505445 7104 raft_consensus.cc:2454] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd0fea80f604c9098ccf443519a118b in term 0.
I20260811 22:20:27.505618 7470 raft_consensus.cc:2781] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:27.505759 7470 raft_consensus.cc:482] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:27.505853 7470 raft_consensus.cc:3037] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:27.507174 7470 raft_consensus.cc:504] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.507649 7470 leader_election.cc:288] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [CANDIDATE]: Term 1 election: Requested vote from peers 37a481dbc37f4c7b9272ead656acedda (127.17.198.1:35237), 7fcb545ee2f04f90a60712789912850f (127.17.198.2:33749)
I20260811 22:20:27.508133 7104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9862a150344dbbaafec39b591e5052" candidate_uuid: "0cd0fea80f604c9098ccf443519a118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a481dbc37f4c7b9272ead656acedda"
I20260811 22:20:27.508227 7251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9862a150344dbbaafec39b591e5052" candidate_uuid: "0cd0fea80f604c9098ccf443519a118b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fcb545ee2f04f90a60712789912850f"
I20260811 22:20:27.508312 7104 raft_consensus.cc:3037] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:27.508371 7251 raft_consensus.cc:3037] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:27.509677 7104 raft_consensus.cc:2454] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd0fea80f604c9098ccf443519a118b in term 1.
I20260811 22:20:27.509788 7251 raft_consensus.cc:2454] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd0fea80f604c9098ccf443519a118b in term 1.
I20260811 22:20:27.510200 7323 leader_election.cc:302] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0cd0fea80f604c9098ccf443519a118b, 37a481dbc37f4c7b9272ead656acedda, 7fcb545ee2f04f90a60712789912850f; no voters:
I20260811 22:20:27.510416 7470 raft_consensus.cc:2781] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:27.510727 7470 raft_consensus.cc:686] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 1 LEADER]: Becoming Leader. State: Replica: 0cd0fea80f604c9098ccf443519a118b, State: Running, Role: LEADER
I20260811 22:20:27.511133 7470 consensus_queue.cc:237] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } }
I20260811 22:20:27.515236 6943 catalog_manager.cc:5495] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cd0fea80f604c9098ccf443519a118b (127.17.198.3). New cstate: current_term: 1 leader_uuid: "0cd0fea80f604c9098ccf443519a118b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd0fea80f604c9098ccf443519a118b" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 40965 } health_report { overall_health: HEALTHY } } }
I20260811 22:20:27.533316 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.526686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.17.198.3:40965)
0811 22:20:27.526892 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.526897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.526914 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:20:27.527108 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.527111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.527172 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.527290 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:27.527564 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.528147 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.529126 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.529390 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.529486 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.529950 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:27.532255 (+ 2305us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:27.532390 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.532921 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.533018 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:20:27.540542 7104 raft_consensus.cc:1261] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 1 FOLLOWER]: Refusing update from remote peer 0cd0fea80f604c9098ccf443519a118b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:27.540711 7251 raft_consensus.cc:1261] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 1 FOLLOWER]: Refusing update from remote peer 0cd0fea80f604c9098ccf443519a118b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:27.541534 7472 consensus_queue.cc:1035] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [LEADER]: Connected to new peer: Peer: permanent_uuid: "37a481dbc37f4c7b9272ead656acedda" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:27.541740 7470 consensus_queue.cc:1035] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fcb545ee2f04f90a60712789912850f" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:27.551043 7478 mvcc.cc:205] Tried to move back new op lower bound from 7317450045595615232 to 7317450045489217536. Current Snapshot: MvccSnapshot[applied={T|T < 7317450045595615232}]
I20260811 22:20:27.552033 7477 mvcc.cc:205] Tried to move back new op lower bound from 7317450045595615232 to 7317450045489217536. Current Snapshot: MvccSnapshot[applied={T|T < 7317450045595615232}]
I20260811 22:20:27.559814 6944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:27.560003 6944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:27.563685 6944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29aee08f8269411286199f8bcbc36489: Sending DeleteTablet for 3 replicas of tablet 9c9862a150344dbbaafec39b591e5052
I20260811 22:20:27.564867 7373 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9862a150344dbbaafec39b591e5052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:27 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:54184
I20260811 22:20:27.564846 7082 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9862a150344dbbaafec39b591e5052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:27 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:37054
I20260811 22:20:27.564987 7227 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c9862a150344dbbaafec39b591e5052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:27 UTC) from {username='kudu', principal='kudu/127.17.198.62@KRBTEST.COM'} at 127.0.0.1:51434
I20260811 22:20:27.565769 7487 tablet_replica.cc:330] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: stopping tablet replica
I20260811 22:20:27.565793 7486 tablet_replica.cc:330] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda: stopping tablet replica
I20260811 22:20:27.566169 7487 raft_consensus.cc:2229] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:27.566329 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7012
I20260811 22:20:27.566232 7486 raft_consensus.cc:2229] T 9c9862a150344dbbaafec39b591e5052 P 37a481dbc37f4c7b9272ead656acedda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:27.567085 7488 tablet_replica.cc:330] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: stopping tablet replica
I20260811 22:20:27.567215 7487 raft_consensus.cc:2258] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:27.568202 7488 raft_consensus.cc:2229] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:27.568651 7488 raft_consensus.cc:2258] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:27.573067 7487 ts_tablet_manager.cc:1905] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:27.575821 7487 ts_tablet_manager.cc:1918] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:27.575958 7487 log.cc:1196] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/9c9862a150344dbbaafec39b591e5052
I20260811 22:20:27.576345 7487 ts_tablet_manager.cc:1939] T 9c9862a150344dbbaafec39b591e5052 P 0cd0fea80f604c9098ccf443519a118b: Deleting consensus metadata
I20260811 22:20:27.577064 7488 ts_tablet_manager.cc:1905] T 9c9862a150344dbbaafec39b591e5052 P 7fcb545ee2f04f90a60712789912850f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:27.577857 6927 catalog_manager.cc:4841] TS 0cd0fea80f604c9098ccf443519a118b (127.17.198.3:40965): tablet 9c9862a150344dbbaafec39b591e5052 (table test-table [id=797ea639d35343b0a8bd4fe085d3d88b]) successfully deleted
W20260811 22:20:27.578341 6929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:35237 (error 108)
I20260811 22:20:27.578435 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7162
I20260811 22:20:27.592504 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.592020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:35237 (local address 127.0.0.1:37062)
0811 22:20:27.592282 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.592373 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260811 22:20:27.592741 6929 catalog_manager.cc:4584] TS 37a481dbc37f4c7b9272ead656acedda (127.17.198.1:35237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9c9862a150344dbbaafec39b591e5052: Network error: Client connection negotiation failed: client connection to 127.17.198.1:35237: connect: Connection refused (error 111)
I20260811 22:20:27.600152 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7305
I20260811 22:20:27.603382 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.600509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:33749 (local address 127.0.0.1:51438)
0811 22:20:27.603151 (+ 2642us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:27.603250 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.2:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2512}
I20260811 22:20:27.608978 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 6911
2026-08-11T22:20:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2601 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:20:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:27Z Disabled control of system clock
I20260811 22:20:27.647639 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:44877
--webserver_interface=127.17.198.62
--webserver_port=0
--builtin_ntp_servers=127.17.198.20:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:44877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:27.768409 7493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:27.768724 7493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:27.768772 7493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:27.773070 7493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:27.773156 7493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:27.773171 7493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:27.773190 7493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:27.773204 7493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:27.778592 7493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:44877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:44877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:27.779834 7493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:27.781329 7493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:27.792455 7493 server_base.cc:1034] running on GCE node
W20260811 22:20:27.792339 7498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:27.792402 7499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:27.792342 7501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:27.793474 7493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:27.794906 7493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:27.796106 7493 hybrid_clock.cc:648] HybridClock initialized: now 1786486827796084 us; error 38 us; skew 500 ppm
I20260811 22:20:27.804651 7493 webserver.cc:460] Webserver started at http://127.17.198.62:38123/ using document root <none> and password file <none>
I20260811 22:20:27.805456 7493 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:27.805552 7493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:27.805752 7493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:27.808260 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "95a22674cbc34aff8bbf1bec24286867"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "3c3e823541bcf545abf78855ac7811a8"
server_key_iv: "4b1115540c6081f7ef8d75e4f6a191b2"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.808980 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "95a22674cbc34aff8bbf1bec24286867"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "3c3e823541bcf545abf78855ac7811a8"
server_key_iv: "4b1115540c6081f7ef8d75e4f6a191b2"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.813257 7493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:20:27.816152 7507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:27.817797 7493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:27.817978 7493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "95a22674cbc34aff8bbf1bec24286867"
format_stamp: "Formatted at 2026-08-11 22:20:27 on dist-test-slave-77v9"
server_key: "3c3e823541bcf545abf78855ac7811a8"
server_key_iv: "4b1115540c6081f7ef8d75e4f6a191b2"
server_key_version: "encryptionkey@0"
I20260811 22:20:27.818126 7493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:27.841979 7493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:27.843227 7493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:27.843487 7493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:27.855011 7493 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:44877
I20260811 22:20:27.855052 7563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:44877 every 8 connection(s)
I20260811 22:20:27.856717 7493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:27.857607 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7493
I20260811 22:20:27.857774 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:27.858126 18200 external_mini_cluster.cc:1428] Setting key 1614a81f6b96df6f81dda27f86523b82
I20260811 22:20:27.861254 7564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:27.870007 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: Bootstrap starting.
I20260811 22:20:27.871959 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:27.860353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.17.198.62:44877)
0811 22:20:27.861121 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:27.861143 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:27.861192 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:20:27.862109 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:27.862135 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:27.862676 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:27.863080 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:27.863570 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.865190 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.866562 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:27.867153 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:27.867340 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:27.869647 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:27.869655 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:27.869675 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:27.870377 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:27.870477 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:27.870507 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:27.870628 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:27.870881 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:20:27.871187 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":165,"threads_started":1}
I20260811 22:20:27.873634 7564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:27.874745 7564 log.cc:824] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:27.877648 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: No bootstrap required, opened a new log
I20260811 22:20:27.884156 7564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } }
I20260811 22:20:27.884579 7564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:27.884686 7564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a22674cbc34aff8bbf1bec24286867, State: Initialized, Role: FOLLOWER
I20260811 22:20:27.885339 7564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } }
I20260811 22:20:27.885540 7564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:27.885623 7564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:27.885756 7564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:27.887430 7564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } }
I20260811 22:20:27.887986 7564 leader_election.cc:302] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95a22674cbc34aff8bbf1bec24286867; no voters:
I20260811 22:20:27.888547 7564 leader_election.cc:288] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:27.888634 7570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:27.888993 7570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [term 1 LEADER]: Becoming Leader. State: Replica: 95a22674cbc34aff8bbf1bec24286867, State: Running, Role: LEADER
I20260811 22:20:27.889340 7570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } }
I20260811 22:20:27.890100 7564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:27.891490 7571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95a22674cbc34aff8bbf1bec24286867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } } }
I20260811 22:20:27.891492 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a22674cbc34aff8bbf1bec24286867. Latest consensus state: current_term: 1 leader_uuid: "95a22674cbc34aff8bbf1bec24286867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a22674cbc34aff8bbf1bec24286867" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 44877 } } }
I20260811 22:20:27.891660 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:27.891660 7571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:27.892172 7579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:27.895470 7579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:27.902531 7579 catalog_manager.cc:1347] Generated new cluster ID: 2627c114916c411e819d174bd27529e4
I20260811 22:20:27.902647 7579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:27.912611 7579 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:27.913883 7579 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:27.930001 7579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: Generated new TSK 0
I20260811 22:20:27.930873 7579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:27.940269 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.1:0
--local_ip_for_outbound_sockets=127.17.198.1
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--builtin_ntp_servers=127.17.198.20:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:20:28.066704 7590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:28.067068 7590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:28.067153 7590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:28.071583 7590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:28.071702 7590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:28.071805 7590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.1
I20260811 22:20:28.076871 7590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.198.1
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.1
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:28.078086 7590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:28.079659 7590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:28.089895 7598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.089977 7595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.090073 7596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:28.090113 7590 server_base.cc:1034] running on GCE node
I20260811 22:20:28.090727 7590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:28.091553 7590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:28.092754 7590 hybrid_clock.cc:648] HybridClock initialized: now 1786486828092704 us; error 68 us; skew 500 ppm
I20260811 22:20:28.094954 7590 webserver.cc:460] Webserver started at http://127.17.198.1:38815/ using document root <none> and password file <none>
I20260811 22:20:28.095659 7590 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:28.095763 7590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:28.096014 7590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:28.098384 7590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data/instance:
uuid: "47974818a1bb496f8000b5685834d49c"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b32bec6694e501a5830ba73937af2232"
server_key_iv: "01d27329f1fc7745e4d3f9cc475ac0c9"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.099025 7590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance:
uuid: "47974818a1bb496f8000b5685834d49c"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b32bec6694e501a5830ba73937af2232"
server_key_iv: "01d27329f1fc7745e4d3f9cc475ac0c9"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.103214 7590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:28.106074 7604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.107494 7590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:28.107658 7590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal
uuid: "47974818a1bb496f8000b5685834d49c"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b32bec6694e501a5830ba73937af2232"
server_key_iv: "01d27329f1fc7745e4d3f9cc475ac0c9"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.107805 7590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:28.123751 7590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:28.124562 7590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:28.124831 7590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:28.125510 7590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:28.126580 7590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:28.126664 7590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.126753 7590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:28.126799 7590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.136673 7590 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.1:37827
I20260811 22:20:28.136703 7724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.1:37827 every 8 connection(s)
I20260811 22:20:28.137792 7590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:28.139716 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7590
I20260811 22:20:28.139873 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:28.140260 18200 external_mini_cluster.cc:1428] Setting key 9901c64cbecf2b8fa9218d131d850818
I20260811 22:20:28.142673 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.2:0
--local_ip_for_outbound_sockets=127.17.198.2
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--builtin_ntp_servers=127.17.198.20:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:28.149709 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.139973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.1:58753 (local address 127.17.198.62:44877)
0811 22:20:28.140146 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.140153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.141362 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 22:20:28.143276 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.143279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.143333 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.143410 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:28.144746 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.145688 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.147434 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.147666 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.147762 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.149209 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:28.149212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:28.149214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:28.149228 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:28.149247 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:28.149255 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:28.149324 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.149527 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.149588 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:20:28.151259 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.140393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44877 (local address 127.17.198.1:58753)
0811 22:20:28.141220 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.141264 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.143005 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.143642 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.143653 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.143780 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:28.144537 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.144564 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.145834 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.145837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.147292 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.147307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.147848 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.147855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.148096 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.148875 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:28.148878 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:28.148944 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:28.149043 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:28.149349 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:28.149362 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:28.149372 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.149536 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.150015 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":124,"threads_started":1}
I20260811 22:20:28.152626 7725 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44877
I20260811 22:20:28.152938 7725 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:28.153529 7725 heartbeater.cc:511] Master 127.17.198.62:44877 requested a full tablet report, sending...
I20260811 22:20:28.155230 7519 ts_manager.cc:194] Registered new tserver with Master: 47974818a1bb496f8000b5685834d49c (127.17.198.1:37827)
I20260811 22:20:28.156395 7519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.1:58753
W20260811 22:20:28.264917 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:28.265232 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:28.265273 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:28.269495 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:28.269572 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:28.269630 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.2
I20260811 22:20:28.274724 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.198.2
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.2
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:28.275983 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:28.277498 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:28.287420 7737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.287420 7735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.287446 7734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:28.287510 7729 server_base.cc:1034] running on GCE node
I20260811 22:20:28.288215 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:28.289036 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:28.290213 7729 hybrid_clock.cc:648] HybridClock initialized: now 1786486828290187 us; error 41 us; skew 500 ppm
I20260811 22:20:28.292500 7729 webserver.cc:460] Webserver started at http://127.17.198.2:42489/ using document root <none> and password file <none>
I20260811 22:20:28.293171 7729 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:28.293247 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:28.293469 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:28.295706 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data/instance:
uuid: "2b097d4ef4b74cdca23a10f55deca148"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b856af1a9fec1873b6fd357f8e3f05f8"
server_key_iv: "4aea3c5936543c898155f6a67ef8e593"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.296294 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance:
uuid: "2b097d4ef4b74cdca23a10f55deca148"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b856af1a9fec1873b6fd357f8e3f05f8"
server_key_iv: "4aea3c5936543c898155f6a67ef8e593"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.300330 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:20:28.303066 7743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.304464 7729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:20:28.304566 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal
uuid: "2b097d4ef4b74cdca23a10f55deca148"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "b856af1a9fec1873b6fd357f8e3f05f8"
server_key_iv: "4aea3c5936543c898155f6a67ef8e593"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.304726 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:28.322995 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:28.323843 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:28.324074 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:28.324751 7729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:28.325847 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:28.325899 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.325973 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:28.326020 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.336038 7729 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.2:46495
I20260811 22:20:28.336081 7868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.2:46495 every 8 connection(s)
I20260811 22:20:28.337172 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:28.341550 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7729
I20260811 22:20:28.341681 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:28.342010 18200 external_mini_cluster.cc:1428] Setting key 927c8530b5c632599cd71f55a4152fd2
I20260811 22:20:28.344501 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.198.3:0
--local_ip_for_outbound_sockets=127.17.198.3
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--builtin_ntp_servers=127.17.198.20:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:20:28.348966 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.339291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:46357 (local address 127.17.198.62:44877)
0811 22:20:28.339463 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.339467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.340682 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 22:20:28.342493 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.342496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.342546 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.342616 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:28.343991 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.344581 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.346080 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.346376 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.346501 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.348179 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:28.348187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:28.348193 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:28.348215 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:28.348245 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:28.348261 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:28.348357 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.348582 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.348676 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:20:28.349939 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.339731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44877 (local address 127.17.198.2:46357)
0811 22:20:28.340504 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.340551 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.342226 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.342847 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.342861 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.342974 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:28.343808 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.343833 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.344831 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.344837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.345929 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.345940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.346670 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.346676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.346940 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.347779 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:28.347781 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:28.347847 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:28.347950 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:28.348400 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:28.348409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:28.348415 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.348548 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.348932 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":125,"threads_started":1}
I20260811 22:20:28.351559 7869 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44877
I20260811 22:20:28.351871 7869 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:28.352514 7869 heartbeater.cc:511] Master 127.17.198.62:44877 requested a full tablet report, sending...
I20260811 22:20:28.353931 7519 ts_manager.cc:194] Registered new tserver with Master: 2b097d4ef4b74cdca23a10f55deca148 (127.17.198.2:46495)
I20260811 22:20:28.355072 7519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.2:46357
W20260811 22:20:28.471989 7872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:28.472283 7872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:28.472325 7872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:28.477411 7872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:28.477505 7872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:28.477576 7872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.198.3
I20260811 22:20:28.483336 7872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.198.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.198.3
--webserver_port=0
--tserver_master_addrs=127.17.198.62:44877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.198.3
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:28.484592 7872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:28.486208 7872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:28.496974 7878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.497001 7879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:28.496974 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:28.497242 7872 server_base.cc:1034] running on GCE node
I20260811 22:20:28.497762 7872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:28.498485 7872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:28.499650 7872 hybrid_clock.cc:648] HybridClock initialized: now 1786486828499622 us; error 44 us; skew 500 ppm
I20260811 22:20:28.501739 7872 webserver.cc:460] Webserver started at http://127.17.198.3:38249/ using document root <none> and password file <none>
I20260811 22:20:28.502344 7872 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:28.502421 7872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:28.502650 7872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:28.504959 7872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data/instance:
uuid: "fd98594eae55473ba0ecc176158b1b39"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "7ac1c65011f3e08a482e7da36363bb74"
server_key_iv: "c20d869b089d95e99ce9d205f8c31a91"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.505566 7872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance:
uuid: "fd98594eae55473ba0ecc176158b1b39"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "7ac1c65011f3e08a482e7da36363bb74"
server_key_iv: "c20d869b089d95e99ce9d205f8c31a91"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.509828 7872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:28.512629 7888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.514093 7872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:20:28.514237 7872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal
uuid: "fd98594eae55473ba0ecc176158b1b39"
format_stamp: "Formatted at 2026-08-11 22:20:28 on dist-test-slave-77v9"
server_key: "7ac1c65011f3e08a482e7da36363bb74"
server_key_iv: "c20d869b089d95e99ce9d205f8c31a91"
server_key_version: "encryptionkey@0"
I20260811 22:20:28.514371 7872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:28.524740 7872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:28.525542 7872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:28.525750 7872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:28.526356 7872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:28.527369 7872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:28.527449 7872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.527534 7872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:28.527581 7872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:28.537123 7872 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.3:38713
I20260811 22:20:28.537187 8011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.3:38713 every 8 connection(s)
I20260811 22:20:28.538082 7872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:28.543757 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 7872
I20260811 22:20:28.543902 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:28.544229 18200 external_mini_cluster.cc:1428] Setting key 50ebec7a3bd9caa0620457894949915e
I20260811 22:20:28.548686 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.540054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.3:41903 (local address 127.17.198.62:44877)
0811 22:20:28.540202 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.540206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.541318 (+ 1112us) server_negotiation.cc:407] Connection header received
0811 22:20:28.542852 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.542856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.542930 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.543043 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:28.543998 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.544744 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.546723 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.546966 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.547052 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.548196 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:28.548199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:28.548201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:28.548214 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:28.548235 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:28.548243 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:28.548305 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.548497 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.548563 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:20:28.549860 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.540396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:44877 (local address 127.17.198.3:41903)
0811 22:20:28.541160 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.541204 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.542622 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.543225 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.543233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.543324 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:28.543831 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.543844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.544954 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.544958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.546575 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.546590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.547116 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.547119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.547282 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.547918 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:20:28.547919 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:28.547966 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:20:28.548041 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:28.548325 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:28.548332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:28.548337 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.548448 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.548708 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260811 22:20:28.551177 8012 heartbeater.cc:347] Connected to a master server at 127.17.198.62:44877
I20260811 22:20:28.551529 8012 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:28.552215 8012 heartbeater.cc:511] Master 127.17.198.62:44877 requested a full tablet report, sending...
I20260811 22:20:28.553483 7519 ts_manager.cc:194] Registered new tserver with Master: fd98594eae55473ba0ecc176158b1b39 (127.17.198.3:38713)
I20260811 22:20:28.554149 7519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.198.3:41903
I20260811 22:20:28.559072 18200 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:28.565428 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.561265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.17.198.62:44877)
0811 22:20:28.561413 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.561419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.561616 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:20:28.561800 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.561804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.561855 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.561990 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:28.562324 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.562850 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.563771 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.563998 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.564104 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.564972 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:28.564976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:28.564983 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:28.564996 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:28.565016 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:28.565023 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:28.565098 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.565206 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.565266 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:20:28.568401 7519 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:28.571484 7519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:28.586894 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.580556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38713 (local address 127.0.0.1:55672)
0811 22:20:28.581194 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.581255 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.581426 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.582348 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.582356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.582382 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:28.582560 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.582570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.583999 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.584003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.585149 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.585160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.586280 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.586288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.586406 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.586446 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.586583 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.586713 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"spinlock_wait_cycles":19712,"thread_start_us":103,"threads_started":1}
I20260811 22:20:28.586927 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.580694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:37827 (local address 127.0.0.1:33542)
0811 22:20:28.581539 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.581555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.581647 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.582628 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.582631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.582647 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:28.582753 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.582760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.584086 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.584090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.585337 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.585347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.586600 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.586603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.586691 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.586711 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.586771 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.586826 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260811 22:20:28.587451 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.581831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.17.198.1:37827)
0811 22:20:28.582162 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.582175 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.582198 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:20:28.582276 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.582284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.582472 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.582910 (+ 438us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:28.583070 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.583880 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.585499 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.586412 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.586600 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.586806 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.586872 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.586997 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":154,"threads_started":1}
I20260811 22:20:28.587658 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.580708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.17.198.3:38713)
0811 22:20:28.581309 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.581326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.581356 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:20:28.581679 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.581688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.581933 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.582128 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:28.582723 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.583865 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.585295 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.586139 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.586269 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.586515 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.587356 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.587456 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":222,"threads_started":1}
I20260811 22:20:28.588119 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.581603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.17.198.2:46495)
0811 22:20:28.581950 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.581966 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.582349 (+ 383us) server_negotiation.cc:407] Connection header received
0811 22:20:28.582609 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.582614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.582785 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.582907 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:28.583335 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.584558 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.585953 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.586878 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.586992 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.587372 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.587794 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.587918 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":119,"threads_started":1}
I20260811 22:20:28.588837 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.581378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.2:46495 (local address 127.0.0.1:55044)
0811 22:20:28.582254 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.582267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.582446 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.583087 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.583089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.583109 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:28.583191 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.583197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.584688 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.584692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.585821 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.585831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.587448 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.587453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.587577 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.587608 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.588538 (+ 930us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.588651 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"spinlock_wait_cycles":4736,"thread_start_us":103,"threads_started":1}
I20260811 22:20:28.590986 7939 tablet_service.cc:1467] Processing CreateTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f (DEFAULT_TABLE table=test-table [id=0fb5c0fa958f4eaeb51c410bbfe16856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:28.590974 7656 tablet_service.cc:1467] Processing CreateTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f (DEFAULT_TABLE table=test-table [id=0fb5c0fa958f4eaeb51c410bbfe16856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:28.592209 7656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 406341b7b9cd42b4b86e15a0b3fe478f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:28.592209 7939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 406341b7b9cd42b4b86e15a0b3fe478f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:28.591898 7799 tablet_service.cc:1467] Processing CreateTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f (DEFAULT_TABLE table=test-table [id=0fb5c0fa958f4eaeb51c410bbfe16856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:28.592904 7799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 406341b7b9cd42b4b86e15a0b3fe478f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:28.599591 8031 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Bootstrap starting.
I20260811 22:20:28.600329 8032 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Bootstrap starting.
I20260811 22:20:28.601145 8033 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Bootstrap starting.
I20260811 22:20:28.602176 8031 tablet_bootstrap.cc:654] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:28.602911 8032 tablet_bootstrap.cc:654] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:28.602937 8031 log.cc:824] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:28.603441 8033 tablet_bootstrap.cc:654] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:28.603801 8032 log.cc:824] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:28.604297 8033 log.cc:824] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:28.605213 8031 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: No bootstrap required, opened a new log
I20260811 22:20:28.605444 8031 ts_tablet_manager.cc:1397] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:20:28.606038 8032 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: No bootstrap required, opened a new log
I20260811 22:20:28.606257 8032 ts_tablet_manager.cc:1397] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:20:28.606498 8033 tablet_bootstrap.cc:492] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: No bootstrap required, opened a new log
I20260811 22:20:28.606731 8033 ts_tablet_manager.cc:1397] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:20:28.609200 8031 raft_consensus.cc:348] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.609504 8031 raft_consensus.cc:374] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:28.609562 8031 raft_consensus.cc:729] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b097d4ef4b74cdca23a10f55deca148, State: Initialized, Role: FOLLOWER
I20260811 22:20:28.610004 8031 consensus_queue.cc:260] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.610690 7869 heartbeater.cc:503] Master 127.17.198.62:44877 was elected leader, sending a full tablet report...
I20260811 22:20:28.611174 8031 ts_tablet_manager.cc:1428] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:20:28.611083 8032 raft_consensus.cc:348] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.611370 8032 raft_consensus.cc:374] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:28.611449 8032 raft_consensus.cc:729] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47974818a1bb496f8000b5685834d49c, State: Initialized, Role: FOLLOWER
I20260811 22:20:28.611549 8033 raft_consensus.cc:348] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.611802 8033 raft_consensus.cc:374] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:28.611856 8033 raft_consensus.cc:729] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd98594eae55473ba0ecc176158b1b39, State: Initialized, Role: FOLLOWER
I20260811 22:20:28.612011 8032 consensus_queue.cc:260] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.612746 7725 heartbeater.cc:503] Master 127.17.198.62:44877 was elected leader, sending a full tablet report...
I20260811 22:20:28.612918 8033 consensus_queue.cc:260] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.613307 8037 raft_consensus.cc:482] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:28.613438 8037 raft_consensus.cc:504] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.614064 8032 ts_tablet_manager.cc:1428] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 22:20:28.614571 8037 leader_election.cc:288] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd98594eae55473ba0ecc176158b1b39 (127.17.198.3:38713), 47974818a1bb496f8000b5685834d49c (127.17.198.1:37827)
I20260811 22:20:28.615626 8033 ts_tablet_manager.cc:1428] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:20:28.615765 8012 heartbeater.cc:503] Master 127.17.198.62:44877 was elected leader, sending a full tablet report...
I20260811 22:20:28.619989 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.615195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:48475 (local address 127.17.198.3:38713)
0811 22:20:28.615327 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.615331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.615343 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:20:28.615389 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.615391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.615438 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.615511 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:28.616183 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.617021 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.618819 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.619365 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.619508 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.619563 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.619770 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.619847 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:20:28.619989 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.614814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.3:38713 (local address 127.17.198.2:48475)
0811 22:20:28.615144 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.615159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.615242 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.615636 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.615640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.615658 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:28.615775 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.615783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.617185 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.617191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.618704 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.618718 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.619504 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.619508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.619618 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.619650 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.619752 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.619847 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260811 22:20:28.620703 7960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "406341b7b9cd42b4b86e15a0b3fe478f" candidate_uuid: "2b097d4ef4b74cdca23a10f55deca148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd98594eae55473ba0ecc176158b1b39" is_pre_election: true
I20260811 22:20:28.621090 7960 raft_consensus.cc:2454] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b097d4ef4b74cdca23a10f55deca148 in term 0.
I20260811 22:20:28.621292 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.614814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:37827 (local address 127.17.198.2:58273)
0811 22:20:28.617015 (+ 2201us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.617029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:28.617164 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:28.617438 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:28.617440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:28.617458 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:28.617600 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.617610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.618704 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.618708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.619648 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:28.619658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.620710 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.620717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:28.620873 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.620913 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:20:28.621043 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:20:28.621125 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2130,"mutex_wait_us":1068,"thread_start_us":133,"threads_started":1}
I20260811 22:20:28.621286 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.614959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.198.2:58273 (local address 127.17.198.1:37827)
0811 22:20:28.615080 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.615086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.617099 (+ 2013us) server_negotiation.cc:407] Connection header received
0811 22:20:28.617270 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.617273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.617347 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.617427 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:28.617720 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.618575 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.619770 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.620310 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.620453 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.620503 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.621083 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.621170 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:20:28.621869 7747 leader_election.cc:302] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b097d4ef4b74cdca23a10f55deca148, fd98594eae55473ba0ecc176158b1b39; no voters:
I20260811 22:20:28.622013 7677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "406341b7b9cd42b4b86e15a0b3fe478f" candidate_uuid: "2b097d4ef4b74cdca23a10f55deca148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47974818a1bb496f8000b5685834d49c" is_pre_election: true
I20260811 22:20:28.622205 8037 raft_consensus.cc:2781] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:28.622325 8037 raft_consensus.cc:482] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:28.622341 7677 raft_consensus.cc:2454] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b097d4ef4b74cdca23a10f55deca148 in term 0.
I20260811 22:20:28.622395 8037 raft_consensus.cc:3037] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:28.623656 8037 raft_consensus.cc:504] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.624012 8037 leader_election.cc:288] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [CANDIDATE]: Term 1 election: Requested vote from peers fd98594eae55473ba0ecc176158b1b39 (127.17.198.3:38713), 47974818a1bb496f8000b5685834d49c (127.17.198.1:37827)
I20260811 22:20:28.624409 7960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "406341b7b9cd42b4b86e15a0b3fe478f" candidate_uuid: "2b097d4ef4b74cdca23a10f55deca148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd98594eae55473ba0ecc176158b1b39"
I20260811 22:20:28.624425 7677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "406341b7b9cd42b4b86e15a0b3fe478f" candidate_uuid: "2b097d4ef4b74cdca23a10f55deca148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47974818a1bb496f8000b5685834d49c"
I20260811 22:20:28.624528 7960 raft_consensus.cc:3037] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:28.624603 7677 raft_consensus.cc:3037] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:28.625634 7960 raft_consensus.cc:2454] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b097d4ef4b74cdca23a10f55deca148 in term 1.
I20260811 22:20:28.625999 7747 leader_election.cc:302] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b097d4ef4b74cdca23a10f55deca148, fd98594eae55473ba0ecc176158b1b39; no voters:
I20260811 22:20:28.626039 7677 raft_consensus.cc:2454] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b097d4ef4b74cdca23a10f55deca148 in term 1.
I20260811 22:20:28.626258 8037 raft_consensus.cc:2781] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:28.626528 8037 raft_consensus.cc:686] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 1 LEADER]: Becoming Leader. State: Replica: 2b097d4ef4b74cdca23a10f55deca148, State: Running, Role: LEADER
I20260811 22:20:28.626961 8037 consensus_queue.cc:237] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } }
I20260811 22:20:28.630354 7519 catalog_manager.cc:5495] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b097d4ef4b74cdca23a10f55deca148 (127.17.198.2). New cstate: current_term: 1 leader_uuid: "2b097d4ef4b74cdca23a10f55deca148" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b097d4ef4b74cdca23a10f55deca148" member_type: VOTER last_known_addr { host: "127.17.198.2" port: 46495 } health_report { overall_health: HEALTHY } } }
W20260811 22:20:28.639341 7726 tablet.cc:2369] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:28.653436 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.649807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.17.198.2:46495)
0811 22:20:28.649955 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:28.649961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:28.649978 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:20:28.650072 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:28.650074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:28.650132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:28.650248 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:28.650495 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.650999 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.651863 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:28.652105 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:28.652212 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:28.652531 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:28.653040 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:28.653174 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:28.653215 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 22:20:28.653275 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:20:28.659996 7960 raft_consensus.cc:1261] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 1 FOLLOWER]: Refusing update from remote peer 2b097d4ef4b74cdca23a10f55deca148: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:28.659989 7677 raft_consensus.cc:1261] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 1 FOLLOWER]: Refusing update from remote peer 2b097d4ef4b74cdca23a10f55deca148: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:28.660800 8042 consensus_queue.cc:1035] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47974818a1bb496f8000b5685834d49c" member_type: VOTER last_known_addr { host: "127.17.198.1" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:28.661152 8037 consensus_queue.cc:1035] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd98594eae55473ba0ecc176158b1b39" member_type: VOTER last_known_addr { host: "127.17.198.3" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:28.671999 8047 mvcc.cc:205] Tried to move back new op lower bound from 7317450050183397376 to 7317450050058973184. Current Snapshot: MvccSnapshot[applied={T|T < 7317450050183397376 or (T in {7317450050183397376})}]
I20260811 22:20:28.680186 7519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:28.680393 7519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:28.683037 7519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95a22674cbc34aff8bbf1bec24286867: Sending DeleteTablet for 3 replicas of tablet 406341b7b9cd42b4b86e15a0b3fe478f
I20260811 22:20:28.683954 7939 tablet_service.cc:1514] Processing DeleteTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:28 UTC) from {username='slave'} at 127.0.0.1:55672
I20260811 22:20:28.684059 7656 tablet_service.cc:1514] Processing DeleteTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:28 UTC) from {username='slave'} at 127.0.0.1:33542
I20260811 22:20:28.684626 8056 tablet_replica.cc:330] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c: stopping tablet replica
I20260811 22:20:28.684679 7799 tablet_service.cc:1514] Processing DeleteTablet for tablet 406341b7b9cd42b4b86e15a0b3fe478f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:28 UTC) from {username='slave'} at 127.0.0.1:55044
I20260811 22:20:28.685045 8056 raft_consensus.cc:2229] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:28.685176 8057 tablet_replica.cc:330] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: stopping tablet replica
I20260811 22:20:28.685446 8056 raft_consensus.cc:2258] T 406341b7b9cd42b4b86e15a0b3fe478f P 47974818a1bb496f8000b5685834d49c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:28.685448 8057 raft_consensus.cc:2229] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:28.685521 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7590
I20260811 22:20:28.686964 8055 tablet_replica.cc:330] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: stopping tablet replica
I20260811 22:20:28.687284 8055 raft_consensus.cc:2229] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:28.687592 8055 raft_consensus.cc:2258] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:28.688109 8057 raft_consensus.cc:2258] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:28.691108 8055 ts_tablet_manager.cc:1905] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:28.692243 8057 ts_tablet_manager.cc:1905] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:28.695134 8057 ts_tablet_manager.cc:1918] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:28.695264 8055 ts_tablet_manager.cc:1918] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:28.695394 8055 log.cc:1196] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-2/wal/wals/406341b7b9cd42b4b86e15a0b3fe478f
I20260811 22:20:28.695393 8057 log.cc:1196] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Deleting WAL directory at /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486470476166-18200-0/minicluster-data/ts-1/wal/wals/406341b7b9cd42b4b86e15a0b3fe478f
I20260811 22:20:28.695750 8057 ts_tablet_manager.cc:1939] T 406341b7b9cd42b4b86e15a0b3fe478f P 2b097d4ef4b74cdca23a10f55deca148: Deleting consensus metadata
I20260811 22:20:28.695791 8055 ts_tablet_manager.cc:1939] T 406341b7b9cd42b4b86e15a0b3fe478f P fd98594eae55473ba0ecc176158b1b39: Deleting consensus metadata
I20260811 22:20:28.697494 7510 catalog_manager.cc:4841] TS fd98594eae55473ba0ecc176158b1b39 (127.17.198.3:38713): tablet 406341b7b9cd42b4b86e15a0b3fe478f (table test-table [id=0fb5c0fa958f4eaeb51c410bbfe16856]) successfully deleted
I20260811 22:20:28.697782 7511 catalog_manager.cc:4841] TS 2b097d4ef4b74cdca23a10f55deca148 (127.17.198.2:46495): tablet 406341b7b9cd42b4b86e15a0b3fe478f (table test-table [id=0fb5c0fa958f4eaeb51c410bbfe16856]) successfully deleted
W20260811 22:20:28.698316 7511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.198.1:37827 (error 108)
I20260811 22:20:28.698928 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7729
I20260811 22:20:28.702167 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:28.699153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.1:37827 (local address 127.0.0.1:33546)
0811 22:20:28.701956 (+ 2803us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:28.702057 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.1:37827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2669}
W20260811 22:20:28.703428 7511 catalog_manager.cc:4584] TS 47974818a1bb496f8000b5685834d49c (127.17.198.1:37827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 406341b7b9cd42b4b86e15a0b3fe478f: Network error: Client connection negotiation failed: client connection to 127.17.198.1:37827: connect: Connection refused (error 111)
I20260811 22:20:28.708963 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7872
I20260811 22:20:28.715835 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 7493
2026-08-11T22:20:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1104 ms)
[----------] 6 tests from EncryptionPolicyTest (11384 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:28 dist-test-slave-77v9 krb5kdc[8061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:28 dist-test-slave-77v9 krb5kdc[8061](info): set up 2 sockets
Aug 11 22:20:28 dist-test-slave-77v9 krb5kdc[8061](info): commencing operation
krb5kdc: starting...
W20260811 22:20:29.894441 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:29 dist-test-slave-77v9 krb5kdc[8061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:30.040777 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:30.161882 8074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:30.162222 8074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:30.162305 8074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:30.166493 8074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:30.166597 8074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:30.166637 8074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:30.166693 8074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:30.166725 8074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:30.172291 8074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:30.173672 8074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:30.174906 8074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:30.184296 8079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:30.184211 8081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:30.184276 8074 server_base.cc:1034] running on GCE node
W20260811 22:20:30.184211 8083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:30.185253 8074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:30.186421 8074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:30.187613 8074 hybrid_clock.cc:648] HybridClock initialized: now 1786486830187564 us; error 66 us; skew 500 ppm
Aug 11 22:20:30 dist-test-slave-77v9 krb5kdc[8061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:30.191037 8074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:30.192101 8074 webserver.cc:460] Webserver started at http://127.0.0.1:34521/ using document root <none> and password file <none>
I20260811 22:20:30.193219 8074 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:30.193308 8074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:30.193594 8074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:30.195869 8074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "48dc21c90c7441b2bea4e1aba7372442"
format_stamp: "Formatted at 2026-08-11 22:20:30 on dist-test-slave-77v9"
server_key: "18309f60a1cc9fdb84595793068e8743"
server_key_iv: "5c193275894a158b683ea2805153db50"
server_key_version: "encryptionkey@0"
I20260811 22:20:30.196486 8074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "48dc21c90c7441b2bea4e1aba7372442"
format_stamp: "Formatted at 2026-08-11 22:20:30 on dist-test-slave-77v9"
server_key: "18309f60a1cc9fdb84595793068e8743"
server_key_iv: "5c193275894a158b683ea2805153db50"
server_key_version: "encryptionkey@0"
I20260811 22:20:30.200968 8074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:20:30.204030 8090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:30.205520 8074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:30.205675 8074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "48dc21c90c7441b2bea4e1aba7372442"
format_stamp: "Formatted at 2026-08-11 22:20:30 on dist-test-slave-77v9"
server_key: "18309f60a1cc9fdb84595793068e8743"
server_key_iv: "5c193275894a158b683ea2805153db50"
server_key_version: "encryptionkey@0"
I20260811 22:20:30.205817 8074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:30.216566 8074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:30.220089 8074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:30.220338 8074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:30.228482 8074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44039
I20260811 22:20:30.228541 8151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44039 every 8 connection(s)
I20260811 22:20:30.229775 8074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:30.232906 8152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:30.238997 8152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442: Bootstrap starting.
I20260811 22:20:30.239547 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8074
I20260811 22:20:30.239715 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:30.240078 18200 external_mini_cluster.cc:1428] Setting key 321ab54a8be6b5f1ae737db92ca4ad69
I20260811 22:20:30.242022 8152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:30.242796 8152 log.cc:824] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:30.245225 8152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442: No bootstrap required, opened a new log
I20260811 22:20:30.249944 8152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260811 22:20:30.250226 8152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:30.250310 8152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dc21c90c7441b2bea4e1aba7372442, State: Initialized, Role: FOLLOWER
Aug 11 22:20:30 dist-test-slave-77v9 krb5kdc[8061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:20:30.250849 8152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260811 22:20:30.251046 8152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:30.251116 8152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:30.251231 8152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:30.252251 8152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260811 22:20:30.252636 8152 leader_election.cc:302] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48dc21c90c7441b2bea4e1aba7372442; no voters:
I20260811 22:20:30.253063 8152 leader_election.cc:288] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:30.253136 8157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:30.253374 8157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [term 1 LEADER]: Becoming Leader. State: Replica: 48dc21c90c7441b2bea4e1aba7372442, State: Running, Role: LEADER
I20260811 22:20:30.253692 8157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260811 22:20:30.254455 8152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:30.255625 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48dc21c90c7441b2bea4e1aba7372442. Latest consensus state: current_term: 1 leader_uuid: "48dc21c90c7441b2bea4e1aba7372442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } } }
I20260811 22:20:30.255757 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:30.256022 8158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48dc21c90c7441b2bea4e1aba7372442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc21c90c7441b2bea4e1aba7372442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } } }
I20260811 22:20:30.256134 8158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:30.256477 8166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:30.258571 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:30.241628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59225 (local address 127.0.0.1:44039)
0811 22:20:30.242160 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:30.242189 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:30.242231 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:20:30.243040 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:30.243060 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:30.243516 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:30.243892 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:30.244876 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:30.245962 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:30.247212 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:30.247727 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:30.247935 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:30.251397 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:30.251405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:30.251420 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:30.251467 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:30.253912 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:30.254547 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:30.254555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:30.254565 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:30.254722 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:30.255333 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:30.255339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:30.255341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:30.255909 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:30.257256 (+ 1347us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:30.257456 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:20:30.257672 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":178,"threads_started":1}
I20260811 22:20:30.260725 8166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:30.261894 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:30.267784 8166 catalog_manager.cc:1347] Generated new cluster ID: 660f40528dad4736bad3046bad8f2209
I20260811 22:20:30.267882 8166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:30.274839 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:30.264296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50071 (local address 127.0.0.1:44039)
0811 22:20:30.264423 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:30.264428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:30.264657 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:20:30.264797 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:30.264800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:30.264882 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:30.265023 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:30.265956 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:30.266578 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:30.267499 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:30.267989 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:30.268109 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:30.269993 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:30.269998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:30.270001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:30.270031 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:30.271863 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:30.272471 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:30.272476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:30.272478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:30.272578 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:30.272966 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:30.272969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:30.272971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:30.273189 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:30.273387 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:30.274381 (+ 994us) server_negotiation.cc:299] Negotiation successful
0811 22:20:30.274505 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:20:30.275458 8166 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:30.276714 8166 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:30.285848 8166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48dc21c90c7441b2bea4e1aba7372442: Generated new TSK 0
I20260811 22:20:30.286628 8166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:30.296079 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8074
W20260811 22:20:30.303969 8059 connection.cc:535] client connection to 127.0.0.1:44039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44039): Connection reset by peer (error 104)
2026-08-11T22:20:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:30 dist-test-slave-77v9 krb5kdc[8184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:30 dist-test-slave-77v9 krb5kdc[8184](info): set up 2 sockets
Aug 11 22:20:30 dist-test-slave-77v9 krb5kdc[8184](info): commencing operation
krb5kdc: starting...
W20260811 22:20:31.493453 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:31 dist-test-slave-77v9 krb5kdc[8184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:31.645051 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:31.763078 8197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:31.763430 8197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:31.763518 8197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:31.767726 8197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:31.767952 8197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:31.768023 8197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:31.768069 8197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:31.768117 8197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:31.773761 8197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:31.775017 8197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:31.776171 8197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:31.785435 8202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:31.785615 8203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:31.785557 8197 server_base.cc:1034] running on GCE node
W20260811 22:20:31.785564 8205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:31.786554 8197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:31.787880 8197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:31.789070 8197 hybrid_clock.cc:648] HybridClock initialized: now 1786486831789039 us; error 43 us; skew 500 ppm
Aug 11 22:20:31 dist-test-slave-77v9 krb5kdc[8184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:31.792436 8197 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:31.793613 8197 webserver.cc:460] Webserver started at http://127.0.0.1:35909/ using document root <none> and password file <none>
I20260811 22:20:31.794270 8197 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:31.794386 8197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:31.794672 8197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:31.797091 8197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "1bbd51b249a649e998193c4540da384f"
format_stamp: "Formatted at 2026-08-11 22:20:31 on dist-test-slave-77v9"
server_key: "6e555fcbe10328b8248d90ef971214f9"
server_key_iv: "939c191dae953767842113ac162283b1"
server_key_version: "encryptionkey@0"
I20260811 22:20:31.797749 8197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "1bbd51b249a649e998193c4540da384f"
format_stamp: "Formatted at 2026-08-11 22:20:31 on dist-test-slave-77v9"
server_key: "6e555fcbe10328b8248d90ef971214f9"
server_key_iv: "939c191dae953767842113ac162283b1"
server_key_version: "encryptionkey@0"
I20260811 22:20:31.802016 8197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:31.805087 8213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:31.806643 8197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:20:31.806807 8197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "1bbd51b249a649e998193c4540da384f"
format_stamp: "Formatted at 2026-08-11 22:20:31 on dist-test-slave-77v9"
server_key: "6e555fcbe10328b8248d90ef971214f9"
server_key_iv: "939c191dae953767842113ac162283b1"
server_key_version: "encryptionkey@0"
I20260811 22:20:31.807008 8197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:31.823786 8197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:31.827521 8197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:31.827761 8197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:31.835955 8270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35503 every 8 connection(s)
I20260811 22:20:31.836027 8197 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35503
I20260811 22:20:31.837080 8197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:31.840111 8271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:31.843415 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8197
I20260811 22:20:31.843587 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:31.843953 18200 external_mini_cluster.cc:1428] Setting key 447f75e1cb2902920ea7bac5bd383ed3
I20260811 22:20:31.846704 8271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f: Bootstrap starting.
I20260811 22:20:31.849107 8271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:31.849907 8271 log.cc:824] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:31.851917 8271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f: No bootstrap required, opened a new log
I20260811 22:20:31.853478 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:31.845541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53941 (local address 127.0.0.1:35503)
0811 22:20:31.846023 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:31.846045 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:31.846081 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:20:31.846750 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:31.846799 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:31.847184 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:31.847536 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:31.847967 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:31.849034 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:31.850044 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:31.850536 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:31.850720 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:31.851687 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:31.851692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:31.851706 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:31.852145 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:31.852203 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:31.852227 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:31.852367 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:31.852544 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 22:20:31.852685 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":146,"threads_started":1}
I20260811 22:20:31.855849 8271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260811 22:20:31.856105 8271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:31.856199 8271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bbd51b249a649e998193c4540da384f, State: Initialized, Role: FOLLOWER
I20260811 22:20:31.856464 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:31.856721 8271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260811 22:20:31.856904 8271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:31.856977 8271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:31.857084 8271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:31.858485 8271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260811 22:20:31.858901 8271 leader_election.cc:302] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bbd51b249a649e998193c4540da384f; no voters:
I20260811 22:20:31.859283 8271 leader_election.cc:288] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:31.859396 8282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:31.859648 8282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [term 1 LEADER]: Becoming Leader. State: Replica: 1bbd51b249a649e998193c4540da384f, State: Running, Role: LEADER
I20260811 22:20:31.860049 8282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260811 22:20:31.860760 8271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:31.861863 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bbd51b249a649e998193c4540da384f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } } }
I20260811 22:20:31.862006 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:31.862388 8284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bbd51b249a649e998193c4540da384f. Latest consensus state: current_term: 1 leader_uuid: "1bbd51b249a649e998193c4540da384f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bbd51b249a649e998193c4540da384f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } } }
I20260811 22:20:31.862560 8284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:31.863539 8292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:31.864217 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:31.858987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44061 (local address 127.0.0.1:35503)
0811 22:20:31.859115 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:31.859120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:31.859360 (+ 240us) server_negotiation.cc:407] Connection header received
0811 22:20:31.859606 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:31.859609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:31.859661 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:31.859738 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:31.860148 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:31.860717 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:31.861806 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:31.862167 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:31.862249 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:31.863408 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:31.863412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:31.863414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:31.863429 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:31.863457 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:31.863470 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:31.863556 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:31.864019 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 22:20:31.864056 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:20:31.867276 8292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:31.873019 8292 catalog_manager.cc:1347] Generated new cluster ID: a0c9da020ce1414bbc4d703449e22946
I20260811 22:20:31.873116 8292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:31.883186 8292 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:31.884325 8292 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:31.894405 8292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bbd51b249a649e998193c4540da384f: Generated new TSK 0
I20260811 22:20:31.895167 8292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:31.904646 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8197
W20260811 22:20:31.914261 8182 connection.cc:535] client connection to 127.0.0.1:35503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35503): Connection reset by peer (error 104)
2026-08-11T22:20:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:31 dist-test-slave-77v9 krb5kdc[8304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:31 dist-test-slave-77v9 krb5kdc[8304](info): set up 2 sockets
Aug 11 22:20:31 dist-test-slave-77v9 krb5kdc[8304](info): commencing operation
krb5kdc: starting...
W20260811 22:20:33.081523 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:33 dist-test-slave-77v9 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:33.224927 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:33.339582 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:33.339933 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:33.340011 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:33.344167 8317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:33.344285 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:33.344322 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:33.344385 8317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:33.344424 8317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:33.350045 8317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42537
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:33.351261 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:33.352340 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:33.361238 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:33.361227 8325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:33.361258 8322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:33.361603 8317 server_base.cc:1034] running on GCE node
I20260811 22:20:33.362207 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:33.363356 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:33.364538 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786486833364519 us; error 33 us; skew 500 ppm
Aug 11 22:20:33 dist-test-slave-77v9 krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:33.373658 8317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:33.374751 8317 webserver.cc:460] Webserver started at http://127.0.0.1:41789/ using document root <none> and password file <none>
I20260811 22:20:33.375344 8317 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:33.375416 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:33.375646 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:33.377947 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:33 on dist-test-slave-77v9"
server_key: "9b916530dd02f3760fc4aa946fe543c9"
server_key_iv: "3b793f8908c4179eb91a678224d3ffe7"
server_key_version: "encryptionkey@0"
I20260811 22:20:33.378577 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:33 on dist-test-slave-77v9"
server_key: "9b916530dd02f3760fc4aa946fe543c9"
server_key_iv: "3b793f8908c4179eb91a678224d3ffe7"
server_key_version: "encryptionkey@0"
I20260811 22:20:33.383109 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:33.385799 8333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:33.387212 8317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:33.387359 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea"
format_stamp: "Formatted at 2026-08-11 22:20:33 on dist-test-slave-77v9"
server_key: "9b916530dd02f3760fc4aa946fe543c9"
server_key_iv: "3b793f8908c4179eb91a678224d3ffe7"
server_key_version: "encryptionkey@0"
I20260811 22:20:33.387516 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:33.392643 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:33.396282 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:33.396505 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:33.404201 8317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42537
I20260811 22:20:33.404229 8388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42537 every 8 connection(s)
I20260811 22:20:33.405211 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:33.408238 8390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:33.412937 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8317
I20260811 22:20:33.413120 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:33.413456 18200 external_mini_cluster.cc:1428] Setting key b1bb4f1af728d95c25ee80be45cf69e3
I20260811 22:20:33.415218 8390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea: Bootstrap starting.
I20260811 22:20:33.417833 8390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:33.418808 8390 log.cc:824] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:33.418905 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:33.415003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46241 (local address 127.0.0.1:42537)
0811 22:20:33.415533 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:33.415548 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:33.415583 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:20:33.416213 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:33.416229 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:33.416501 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:33.416860 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:33.417710 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:33.417715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:33.417728 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:33.417738 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:33.417795 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:33.417813 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:33.417936 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:33.418047 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 22:20:33.418210 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":158,"threads_started":1}
I20260811 22:20:33.421226 8390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea: No bootstrap required, opened a new log
I20260811 22:20:33.421943 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:33.425603 8390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } }
I20260811 22:20:33.425900 8390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:33.426076 8390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b4cd9323dff46ab8de0fea9eb84c4ea, State: Initialized, Role: FOLLOWER
I20260811 22:20:33.426080 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:33.424103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57733 (local address 127.0.0.1:42537)
0811 22:20:33.424242 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:33.424248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:33.424478 (+ 230us) server_negotiation.cc:407] Connection header received
0811 22:20:33.424632 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:33.424635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:33.424668 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:33.424755 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:33.425572 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:33.425575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:33.425578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:33.425589 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:33.425617 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:33.425636 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:33.425728 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:33.425835 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:20:33.425881 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:20:33.426662 8390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } }
I20260811 22:20:33.426882 8390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:33.427027 8390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:33.427157 8390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:33.428735 8390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } }
I20260811 22:20:33.429212 8390 leader_election.cc:302] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b4cd9323dff46ab8de0fea9eb84c4ea; no voters:
I20260811 22:20:33.429637 8390 leader_election.cc:288] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:33.429756 8400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:33.430022 8400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [term 1 LEADER]: Becoming Leader. State: Replica: 0b4cd9323dff46ab8de0fea9eb84c4ea, State: Running, Role: LEADER
I20260811 22:20:33.430404 8400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } }
I20260811 22:20:33.431255 8390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:33.431525 8400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4cd9323dff46ab8de0fea9eb84c4ea. Latest consensus state: current_term: 1 leader_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } } }
I20260811 22:20:33.431670 8400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:33.433502 8407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:33.433688 8404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4cd9323dff46ab8de0fea9eb84c4ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42537 } } }
I20260811 22:20:33.433815 8404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:33.438033 8407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:33.445820 8407 catalog_manager.cc:1347] Generated new cluster ID: 0f804bb85a744390972775fdf9e8efe2
I20260811 22:20:33.445920 8407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:33.456508 8407 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:33.457005 8407 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:33.473452 8407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b4cd9323dff46ab8de0fea9eb84c4ea: Generated new TSK 0
I20260811 22:20:33.474393 8407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:33.485494 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8317
2026-08-11T22:20:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1581 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:33 dist-test-slave-77v9 krb5kdc[8426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:33 dist-test-slave-77v9 krb5kdc[8426](info): set up 2 sockets
Aug 11 22:20:33 dist-test-slave-77v9 krb5kdc[8426](info): commencing operation
krb5kdc: starting...
W20260811 22:20:34.663105 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:34 dist-test-slave-77v9 krb5kdc[8426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:34.812983 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:34.930051 8439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:34.930359 8439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:34.930413 8439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:34.934614 8439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:34.934696 8439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:34.934717 8439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:34.934734 8439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:34.934749 8439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:34.940763 8439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:34.942226 8439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:34.943252 8439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:34.953372 8444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:34.953372 8445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:34.953374 8447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:34.953677 8439 server_base.cc:1034] running on GCE node
I20260811 22:20:34.954309 8439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:34.955881 8439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:34.957098 8439 hybrid_clock.cc:648] HybridClock initialized: now 1786486834957052 us; error 62 us; skew 500 ppm
Aug 11 22:20:34 dist-test-slave-77v9 krb5kdc[8426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:34.960287 8439 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:34.961421 8439 webserver.cc:460] Webserver started at http://127.0.0.1:34677/ using document root <none> and password file <none>
I20260811 22:20:34.962069 8439 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:34.962174 8439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:34.962437 8439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:34.964663 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c3791dd9bb914d729316c15ee2b96a49"
format_stamp: "Formatted at 2026-08-11 22:20:34 on dist-test-slave-77v9"
server_key: "2fd746bc7a26c4ac3c24db00bac8e8b4"
server_key_iv: "5d2071ab523f82d715c6a2fdbb7648d0"
server_key_version: "encryptionkey@0"
I20260811 22:20:34.965319 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c3791dd9bb914d729316c15ee2b96a49"
format_stamp: "Formatted at 2026-08-11 22:20:34 on dist-test-slave-77v9"
server_key: "2fd746bc7a26c4ac3c24db00bac8e8b4"
server_key_iv: "5d2071ab523f82d715c6a2fdbb7648d0"
server_key_version: "encryptionkey@0"
I20260811 22:20:34.969408 8439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:34.972051 8454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:34.973534 8439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:34.973722 8439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c3791dd9bb914d729316c15ee2b96a49"
format_stamp: "Formatted at 2026-08-11 22:20:34 on dist-test-slave-77v9"
server_key: "2fd746bc7a26c4ac3c24db00bac8e8b4"
server_key_iv: "5d2071ab523f82d715c6a2fdbb7648d0"
server_key_version: "encryptionkey@0"
I20260811 22:20:34.973857 8439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:34.985705 8439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:34.989715 8439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:34.989981 8439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:34.997906 8439 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42977
I20260811 22:20:34.997931 8510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42977 every 8 connection(s)
I20260811 22:20:34.999006 8439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:35.001572 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8439
I20260811 22:20:35.001729 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:35.002058 18200 external_mini_cluster.cc:1428] Setting key 05fd6c96500cee86160ef12a90e2c29e
I20260811 22:20:35.002254 8511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:35.010058 8511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: Bootstrap starting.
I20260811 22:20:35.012871 8511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: Neither blocks nor log segments found. Creating new log.
Aug 11 22:20:35 dist-test-slave-77v9 krb5kdc[8426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:20:35.013962 8511 log.cc:824] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:35.016712 8511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: No bootstrap required, opened a new log
I20260811 22:20:35.020288 8514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:35.003601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36291 (local address 127.0.0.1:42977)
0811 22:20:35.004109 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:35.004129 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:35.004179 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:20:35.005252 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:35.005278 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:35.005733 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:35.006152 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:35.007055 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:35.008615 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:35.009725 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:35.010437 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:35.010599 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:35.013703 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:35.013709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:35.013734 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:35.013768 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:35.016320 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:35.016969 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:35.016977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:35.016984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:35.017177 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:35.017524 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:35.017528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:35.017530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:35.018135 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:35.018345 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:35.018674 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:20:35.019098 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":173,"threads_started":1}
I20260811 22:20:35.022231 8511 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260811 22:20:35.022605 8511 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:35.022676 8511 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3791dd9bb914d729316c15ee2b96a49, State: Initialized, Role: FOLLOWER
I20260811 22:20:35.023224 8511 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260811 22:20:35.023473 8511 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:35.023574 8511 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:35.023691 8511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:35.024021 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:35.025048 8511 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260811 22:20:35.025463 8511 leader_election.cc:302] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3791dd9bb914d729316c15ee2b96a49; no voters:
I20260811 22:20:35.025856 8511 leader_election.cc:288] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:35.026017 8520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:35.026268 8520 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [term 1 LEADER]: Becoming Leader. State: Replica: c3791dd9bb914d729316c15ee2b96a49, State: Running, Role: LEADER
I20260811 22:20:35.026628 8520 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260811 22:20:35.027606 8511 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:35.028566 8521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3791dd9bb914d729316c15ee2b96a49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } } }
I20260811 22:20:35.028676 8521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:35.028519 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3791dd9bb914d729316c15ee2b96a49. Latest consensus state: current_term: 1 leader_uuid: "c3791dd9bb914d729316c15ee2b96a49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3791dd9bb914d729316c15ee2b96a49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } } }
I20260811 22:20:35.028767 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49 [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:35.033943 8539 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:35.034049 8539 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:35.034149 8532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:35.035208 8532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:35.038089 8514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:35.026638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41057 (local address 127.0.0.1:42977)
0811 22:20:35.026768 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:35.026773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:35.027443 (+ 670us) server_negotiation.cc:407] Connection header received
0811 22:20:35.027658 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:35.027662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:35.027725 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:35.027819 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:35.029043 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:35.029993 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:35.031213 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:35.031558 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:35.031673 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:35.033821 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:35.033825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:35.033828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:35.033856 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:35.035945 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:35.036567 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:35.036571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:35.036573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:35.036671 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:35.037106 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:35.037109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:35.037111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:35.037357 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:35.037537 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:35.037793 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:20:35.037919 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:20:35.042445 8532 catalog_manager.cc:1347] Generated new cluster ID: 04349c107c9f423fb41181b7d6580d84
I20260811 22:20:35.042542 8532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:35.060559 8532 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:35.062088 8532 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:35.068545 8532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3791dd9bb914d729316c15ee2b96a49: Generated new TSK 0
I20260811 22:20:35.069368 8532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:35.075024 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8439
2026-08-11T22:20:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:35 dist-test-slave-77v9 krb5kdc[8545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:35 dist-test-slave-77v9 krb5kdc[8545](info): set up 2 sockets
Aug 11 22:20:35 dist-test-slave-77v9 krb5kdc[8545](info): commencing operation
krb5kdc: starting...
W20260811 22:20:36.244752 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:36 dist-test-slave-77v9 krb5kdc[8545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:36.385043 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:36.503194 8561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:36.503558 8561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:36.503640 8561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:36.508024 8561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:36.508160 8561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:36.508199 8561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:36.508242 8561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:36.508275 8561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:36.514252 8561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:36.515515 8561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:36.516631 8561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:36.526366 8567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:36.526436 8568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:36.526554 8570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:36.526846 8561 server_base.cc:1034] running on GCE node
I20260811 22:20:36.527487 8561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:36.528690 8561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:36.529881 8561 hybrid_clock.cc:648] HybridClock initialized: now 1786486836529845 us; error 54 us; skew 500 ppm
Aug 11 22:20:36 dist-test-slave-77v9 krb5kdc[8545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:36.533231 8561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:36.534296 8561 webserver.cc:460] Webserver started at http://127.0.0.1:40051/ using document root <none> and password file <none>
I20260811 22:20:36.534899 8561 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:36.535000 8561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:36.535259 8561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:36.537540 8561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3"
format_stamp: "Formatted at 2026-08-11 22:20:36 on dist-test-slave-77v9"
server_key: "08950593a4eccca8c7d784813ff87351"
server_key_iv: "e49131741420eab1136818810357f459"
server_key_version: "encryptionkey@0"
I20260811 22:20:36.538163 8561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3"
format_stamp: "Formatted at 2026-08-11 22:20:36 on dist-test-slave-77v9"
server_key: "08950593a4eccca8c7d784813ff87351"
server_key_iv: "e49131741420eab1136818810357f459"
server_key_version: "encryptionkey@0"
I20260811 22:20:36.542217 8561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:36.545116 8577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:36.546612 8561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:36.546804 8561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3"
format_stamp: "Formatted at 2026-08-11 22:20:36 on dist-test-slave-77v9"
server_key: "08950593a4eccca8c7d784813ff87351"
server_key_iv: "e49131741420eab1136818810357f459"
server_key_version: "encryptionkey@0"
I20260811 22:20:36.546952 8561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:36.559296 8561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:36.562922 8561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:36.563488 8561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:36.572465 8561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43511
I20260811 22:20:36.572489 8631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43511 every 8 connection(s)
I20260811 22:20:36.573565 8561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:36.576619 8632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:36.582723 8632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: Bootstrap starting.
I20260811 22:20:36.583418 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8561
I20260811 22:20:36.583611 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:36.583941 18200 external_mini_cluster.cc:1428] Setting key 22bf2fb98ec6e682edfdaeab15d2597b
I20260811 22:20:36.585829 8632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:36.586630 8632 log.cc:824] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:36.589076 8632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: No bootstrap required, opened a new log
I20260811 22:20:36.593104 8632 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:20:36.593374 8632 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:36.593436 8632 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1673fffdec4bb6b1d3dad11d6d98b3, State: Initialized, Role: FOLLOWER
I20260811 22:20:36.593647 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:36.585371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48207 (local address 127.0.0.1:43511)
0811 22:20:36.585902 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:36.585935 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:36.585989 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:20:36.586690 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:36.586737 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:36.587187 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:36.587579 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:36.588040 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:36.589116 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:36.590061 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:36.590540 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:36.590732 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:36.591673 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:36.591678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:36.591691 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:36.592105 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:36.592163 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:36.592187 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:36.592299 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:36.592458 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:20:36.592690 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":153,"threads_started":1}
I20260811 22:20:36.593971 8632 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:20:36.594143 8632 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:36.594210 8632 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:36.594308 8632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:36.595417 8632 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:20:36.595824 8632 leader_election.cc:302] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc1673fffdec4bb6b1d3dad11d6d98b3; no voters:
I20260811 22:20:36.596210 8632 leader_election.cc:288] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:36.596302 8637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:36.596575 8637 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [term 1 LEADER]: Becoming Leader. State: Replica: bc1673fffdec4bb6b1d3dad11d6d98b3, State: Running, Role: LEADER
I20260811 22:20:36.596930 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:36.596951 8637 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:20:36.597674 8632 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:36.598788 8640 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc1673fffdec4bb6b1d3dad11d6d98b3. Latest consensus state: current_term: 1 leader_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } } }
I20260811 22:20:36.598946 8640 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:36.599289 8638 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1673fffdec4bb6b1d3dad11d6d98b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } } }
I20260811 22:20:36.599395 8638 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3 [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:36.603811 8659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:36.603927 8659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:36.603987 8653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:36.604945 8653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:36.605195 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:36.600761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46375 (local address 127.0.0.1:43511)
0811 22:20:36.600901 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:36.600906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:36.601179 (+ 273us) server_negotiation.cc:407] Connection header received
0811 22:20:36.601351 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:36.601354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:36.601402 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:36.601495 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:36.601937 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:36.602472 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:36.603344 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:36.603601 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:36.603721 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:36.604689 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:36.604693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:36.604696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:36.604709 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:36.604747 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:36.604757 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:36.604873 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:36.605001 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 22:20:36.605057 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 22:20:36.611302 8653 catalog_manager.cc:1347] Generated new cluster ID: c7a8da1dec534ddfaa82304d277670c8
I20260811 22:20:36.611397 8653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:36.625402 8653 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:36.626464 8653 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:36.632614 8653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc1673fffdec4bb6b1d3dad11d6d98b3: Generated new TSK 0
I20260811 22:20:36.633570 8653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:36.643697 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8561
2026-08-11T22:20:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:36 dist-test-slave-77v9 krb5kdc[8668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:36 dist-test-slave-77v9 krb5kdc[8668](info): set up 2 sockets
Aug 11 22:20:36 dist-test-slave-77v9 krb5kdc[8668](info): commencing operation
krb5kdc: starting...
W20260811 22:20:37.841984 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:37 dist-test-slave-77v9 krb5kdc[8668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:38.002035 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:38.121804 8682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:38.122103 8682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:38.122149 8682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:38.126410 8682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:38.126518 8682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:38.126538 8682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:38.126556 8682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:38.126572 8682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:38.132762 8682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:38.133962 8682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:38.134979 8682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:38.144280 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:38.144325 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:38.144280 8689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:38.144495 8682 server_base.cc:1034] running on GCE node
I20260811 22:20:38.145380 8682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:38.146589 8682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:38.147769 8682 hybrid_clock.cc:648] HybridClock initialized: now 1786486838147733 us; error 48 us; skew 500 ppm
Aug 11 22:20:38 dist-test-slave-77v9 krb5kdc[8668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:38.151017 8682 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:38.152053 8682 webserver.cc:460] Webserver started at http://127.0.0.1:43537/ using document root <none> and password file <none>
I20260811 22:20:38.152627 8682 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:38.152693 8682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:38.152959 8682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:38.155315 8682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c4be63ba06db42ffb0b15785ff62b6f3"
format_stamp: "Formatted at 2026-08-11 22:20:38 on dist-test-slave-77v9"
server_key: "3e98a5fc3db5b9615987ffe0558519a9"
server_key_iv: "df8c25180b09e243683143e2dab0a2f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:38.155906 8682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c4be63ba06db42ffb0b15785ff62b6f3"
format_stamp: "Formatted at 2026-08-11 22:20:38 on dist-test-slave-77v9"
server_key: "3e98a5fc3db5b9615987ffe0558519a9"
server_key_iv: "df8c25180b09e243683143e2dab0a2f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:38.160485 8682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:20:38.164254 8699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:38.165786 8682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:20:38.165964 8682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c4be63ba06db42ffb0b15785ff62b6f3"
format_stamp: "Formatted at 2026-08-11 22:20:38 on dist-test-slave-77v9"
server_key: "3e98a5fc3db5b9615987ffe0558519a9"
server_key_iv: "df8c25180b09e243683143e2dab0a2f8"
server_key_version: "encryptionkey@0"
I20260811 22:20:38.166108 8682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:38.171577 8682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:38.175436 8682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:38.175698 8682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:38.183987 8682 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36743
I20260811 22:20:38.184015 8754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36743 every 8 connection(s)
I20260811 22:20:38.185189 8682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:38.188041 8756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:38.190618 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8682
I20260811 22:20:38.190757 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:38.191119 18200 external_mini_cluster.cc:1428] Setting key 14b28fd6179f934b73add5ca7faf3383
I20260811 22:20:38.194382 8756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3: Bootstrap starting.
I20260811 22:20:38.197126 8756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:38.197367 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:38.192707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56327 (local address 127.0.0.1:36743)
0811 22:20:38.193263 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:38.193284 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:38.193328 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:20:38.194140 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:38.194163 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:38.194524 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:38.194895 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:38.195877 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:38.195886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:38.195904 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:38.195918 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:38.196020 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:38.196044 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:38.196213 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:38.196290 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 22:20:38.196456 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":138,"threads_started":1}
I20260811 22:20:38.197954 8756 log.cc:824] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:38.200359 8756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3: No bootstrap required, opened a new log
I20260811 22:20:38.200505 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:38.204898 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:38.202990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56589 (local address 127.0.0.1:36743)
0811 22:20:38.203139 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:38.203145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:38.203308 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:20:38.203431 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:38.203435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:38.203476 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:38.203559 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:38.204436 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:38.204439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:38.204444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:38.204458 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:38.204488 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:38.204505 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:38.204604 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:38.204678 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:20:38.204719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:20:38.205502 8756 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } }
I20260811 22:20:38.205760 8756 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:38.205830 8756 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4be63ba06db42ffb0b15785ff62b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:20:38.206362 8756 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } }
I20260811 22:20:38.206519 8756 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:38.206565 8756 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:38.206635 8756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:38.207805 8756 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } }
I20260811 22:20:38.208213 8756 leader_election.cc:302] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4be63ba06db42ffb0b15785ff62b6f3; no voters:
I20260811 22:20:38.208582 8756 leader_election.cc:288] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:38.208746 8767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:38.209111 8767 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [term 1 LEADER]: Becoming Leader. State: Replica: c4be63ba06db42ffb0b15785ff62b6f3, State: Running, Role: LEADER
I20260811 22:20:38.209476 8767 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } }
I20260811 22:20:38.210095 8756 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:38.211264 8768 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } } }
I20260811 22:20:38.211406 8768 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:38.211774 8776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:38.212173 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4be63ba06db42ffb0b15785ff62b6f3. Latest consensus state: current_term: 1 leader_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be63ba06db42ffb0b15785ff62b6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36743 } } }
I20260811 22:20:38.212287 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:38.215600 8776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:38.223272 8776 catalog_manager.cc:1347] Generated new cluster ID: dee217930f764c63ab6ff69b44fdf935
I20260811 22:20:38.223385 8776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:38.252413 8776 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:38.252876 8776 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:38.261109 8776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4be63ba06db42ffb0b15785ff62b6f3: Generated new TSK 0
I20260811 22:20:38.261920 8776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:38.270064 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8682
2026-08-11T22:20:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:38 dist-test-slave-77v9 krb5kdc[8789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:38 dist-test-slave-77v9 krb5kdc[8789](info): set up 2 sockets
Aug 11 22:20:38 dist-test-slave-77v9 krb5kdc[8789](info): commencing operation
krb5kdc: starting...
W20260811 22:20:39.450868 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:39.597247 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:39.720708 8804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:39.721050 8804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:39.721094 8804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:39.725147 8804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:39.725211 8804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:39.725230 8804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:39.725248 8804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:39.725265 8804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:39.730974 8804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:39.732076 8804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:39.733127 8804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:39.742506 8812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:39.742512 8814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:39.742515 8810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:39.742767 8804 server_base.cc:1034] running on GCE node
I20260811 22:20:39.743537 8804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:39.744944 8804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:39.746150 8804 hybrid_clock.cc:648] HybridClock initialized: now 1786486839746135 us; error 31 us; skew 500 ppm
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:39.756249 8804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:39.757565 8804 webserver.cc:460] Webserver started at http://127.0.0.1:42289/ using document root <none> and password file <none>
I20260811 22:20:39.758256 8804 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:39.758373 8804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:39.758647 8804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:39.761184 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "888496b51caf49bebc0020e54ae3dcf6"
format_stamp: "Formatted at 2026-08-11 22:20:39 on dist-test-slave-77v9"
server_key: "39329fabafbf6599aea3ceee66f4e78e"
server_key_iv: "f7de55eb33e00b09e2cac8f456c177cc"
server_key_version: "encryptionkey@0"
I20260811 22:20:39.761798 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "888496b51caf49bebc0020e54ae3dcf6"
format_stamp: "Formatted at 2026-08-11 22:20:39 on dist-test-slave-77v9"
server_key: "39329fabafbf6599aea3ceee66f4e78e"
server_key_iv: "f7de55eb33e00b09e2cac8f456c177cc"
server_key_version: "encryptionkey@0"
I20260811 22:20:39.766089 8804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:20:39.769122 8823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:39.770656 8804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:20:39.770846 8804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "888496b51caf49bebc0020e54ae3dcf6"
format_stamp: "Formatted at 2026-08-11 22:20:39 on dist-test-slave-77v9"
server_key: "39329fabafbf6599aea3ceee66f4e78e"
server_key_iv: "f7de55eb33e00b09e2cac8f456c177cc"
server_key_version: "encryptionkey@0"
I20260811 22:20:39.771051 8804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:39.781633 8804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:39.785508 8804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:39.785763 8804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:39.793429 8804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36419
I20260811 22:20:39.793457 8879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36419 every 8 connection(s)
I20260811 22:20:39.794482 8804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:39.795977 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8804
I20260811 22:20:39.796166 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:39.796515 18200 external_mini_cluster.cc:1428] Setting key 1318b58185954fb38489e4c44cdecda4
I20260811 22:20:39.797847 8880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:39.804786 8880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: Bootstrap starting.
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:20:39.807549 8880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:39.808458 8880 log.cc:824] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:39.810794 8880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: No bootstrap required, opened a new log
I20260811 22:20:39.812649 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:39.798117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:54925 (local address 127.0.0.1:36419)
0811 22:20:39.798569 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:39.798586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:39.798625 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:20:39.799303 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:39.799320 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:39.799657 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:39.799944 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:39.801000 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:39.802009 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:39.803228 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:39.803696 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:39.803830 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:39.806992 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:39.806997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:39.807011 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:39.807041 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:39.809251 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:39.809898 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:39.809902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:39.809907 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:39.810024 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:39.810399 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:39.810402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:39.810404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:39.810815 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:39.811135 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:39.811471 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:20:39.811705 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":168,"threads_started":1}
I20260811 22:20:39.814342 8880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } }
I20260811 22:20:39.814662 8880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:39.814742 8880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888496b51caf49bebc0020e54ae3dcf6, State: Initialized, Role: FOLLOWER
I20260811 22:20:39.815197 8880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } }
I20260811 22:20:39.815358 8880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:39.815431 8880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:39.815538 8880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:39.816421 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:39.816927 8880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } }
I20260811 22:20:39.817282 8880 leader_election.cc:302] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 888496b51caf49bebc0020e54ae3dcf6; no voters:
I20260811 22:20:39.817631 8880 leader_election.cc:288] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:39.817728 8885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:39.817910 8885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [term 1 LEADER]: Becoming Leader. State: Replica: 888496b51caf49bebc0020e54ae3dcf6, State: Running, Role: LEADER
I20260811 22:20:39.818248 8885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } }
I20260811 22:20:39.819020 8880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:39.820142 8886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "888496b51caf49bebc0020e54ae3dcf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } } }
I20260811 22:20:39.820173 8887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 888496b51caf49bebc0020e54ae3dcf6. Latest consensus state: current_term: 1 leader_uuid: "888496b51caf49bebc0020e54ae3dcf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888496b51caf49bebc0020e54ae3dcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36419 } } }
I20260811 22:20:39.820330 8887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:39.820327 8886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6 [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:39.824195 8911 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:39.824321 8911 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:39.824409 8904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:39.825641 8904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:39.831102 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:39.819306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:53271 (local address 127.0.0.1:36419)
0811 22:20:39.819438 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:39.819443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:39.820882 (+ 1439us) server_negotiation.cc:407] Connection header received
0811 22:20:39.821128 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:39.821132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:39.821188 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:39.821249 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:39.822396 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:39.823177 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:39.824518 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:39.824801 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:39.824955 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:39.827078 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:39.827081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:39.827084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:39.827110 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:39.828996 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:39.829605 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:39.829609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:39.829611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:39.829701 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:39.830083 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:39.830086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:39.830088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:39.830290 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:39.830461 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:39.830790 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:20:39.830915 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:20:39.833520 8904 catalog_manager.cc:1347] Generated new cluster ID: d604e7568ad6427fb3a74c6514e2c51a
I20260811 22:20:39.833618 8904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:39.845829 8904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:39.847298 8904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:39.858395 8904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 888496b51caf49bebc0020e54ae3dcf6: Generated new TSK 0
I20260811 22:20:39.859227 8904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:39.872223 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8804
W20260811 22:20:39.882180 8786 connection.cc:535] client connection to 127.0.0.1:36419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36419): Connection reset by peer (error 104)
2026-08-11T22:20:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8917](info): set up 2 sockets
Aug 11 22:20:39 dist-test-slave-77v9 krb5kdc[8917](info): commencing operation
krb5kdc: starting...
W20260811 22:20:40.999637 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:41 dist-test-slave-77v9 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:41.141076 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:41.268035 8930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:41.268405 8930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:41.268497 8930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:41.272718 8930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:41.272879 8930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:41.272924 8930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:41.272984 8930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:41.273038 8930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:41.278740 8930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:41.279958 8930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:41.281054 8930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:41.290416 8937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:41.290534 8930 server_base.cc:1034] running on GCE node
W20260811 22:20:41.290426 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:41.290462 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:41.291306 8930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:41.292512 8930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:41.293690 8930 hybrid_clock.cc:648] HybridClock initialized: now 1786486841293703 us; error 91 us; skew 500 ppm
Aug 11 22:20:41 dist-test-slave-77v9 krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:41.296795 8930 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:41.297986 8930 webserver.cc:460] Webserver started at http://127.0.0.1:40099/ using document root <none> and password file <none>
I20260811 22:20:41.298578 8930 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:41.298674 8930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:41.298919 8930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:41.301168 8930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "e35fd53df4bb4a2ea3857b7df6460472"
format_stamp: "Formatted at 2026-08-11 22:20:41 on dist-test-slave-77v9"
server_key: "d0b4520297c4bdb66a55703225a8cf40"
server_key_iv: "714ff46bf452310ab5df9d1c30403896"
server_key_version: "encryptionkey@0"
I20260811 22:20:41.301818 8930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "e35fd53df4bb4a2ea3857b7df6460472"
format_stamp: "Formatted at 2026-08-11 22:20:41 on dist-test-slave-77v9"
server_key: "d0b4520297c4bdb66a55703225a8cf40"
server_key_iv: "714ff46bf452310ab5df9d1c30403896"
server_key_version: "encryptionkey@0"
I20260811 22:20:41.305903 8930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:20:41.308542 8948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:41.310031 8930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:20:41.310225 8930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "e35fd53df4bb4a2ea3857b7df6460472"
format_stamp: "Formatted at 2026-08-11 22:20:41 on dist-test-slave-77v9"
server_key: "d0b4520297c4bdb66a55703225a8cf40"
server_key_iv: "714ff46bf452310ab5df9d1c30403896"
server_key_version: "encryptionkey@0"
I20260811 22:20:41.310371 8930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:41.322962 8930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:41.326700 8930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:41.326954 8930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:41.335047 8930 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42655
I20260811 22:20:41.335073 9003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42655 every 8 connection(s)
I20260811 22:20:41.336092 8930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:41.339159 9004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:41.339670 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 8930
I20260811 22:20:41.339825 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:41.340157 18200 external_mini_cluster.cc:1428] Setting key fa9e7828bdee979c407f5a180f82e56a
I20260811 22:20:41.346616 9004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472: Bootstrap starting.
I20260811 22:20:41.349327 9004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:41.349610 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:41.341726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:40929 (local address 127.0.0.1:42655)
0811 22:20:41.342234 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:41.342249 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:41.342284 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:20:41.342998 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:41.343015 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:41.343362 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:41.343716 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:41.344126 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:41.345189 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:41.346112 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:41.346609 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:41.346788 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:41.347795 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:41.347799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:41.347812 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:41.348208 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:41.348264 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:41.348287 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:41.348429 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:41.348615 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 22:20:41.348759 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260811 22:20:41.350335 9004 log.cc:824] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:41.352908 9004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472: No bootstrap required, opened a new log
I20260811 22:20:41.353037 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:41.357359 9004 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } }
I20260811 22:20:41.357667 9004 raft_consensus.cc:374] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:41.357748 9004 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35fd53df4bb4a2ea3857b7df6460472, State: Initialized, Role: FOLLOWER
I20260811 22:20:41.358282 9004 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } }
I20260811 22:20:41.358501 9004 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:41.358618 9004 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:41.358754 9004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:41.359583 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:41.355147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:55039 (local address 127.0.0.1:42655)
0811 22:20:41.355283 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:41.355287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:41.355516 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:20:41.355676 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:41.355679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:41.355734 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:41.355819 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:41.356107 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:41.356646 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:41.357538 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:41.357918 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:41.358038 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:41.359091 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:41.359095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:41.359098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:41.359114 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:41.359149 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:41.359160 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:41.359261 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:41.359385 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:20:41.359423 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:20:41.360201 9004 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } }
I20260811 22:20:41.360656 9004 leader_election.cc:302] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35fd53df4bb4a2ea3857b7df6460472; no voters:
I20260811 22:20:41.361152 9004 leader_election.cc:288] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:41.361231 9016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:41.361537 9016 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [term 1 LEADER]: Becoming Leader. State: Replica: e35fd53df4bb4a2ea3857b7df6460472, State: Running, Role: LEADER
I20260811 22:20:41.361871 9016 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } }
I20260811 22:20:41.362658 9004 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:41.363633 9018 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35fd53df4bb4a2ea3857b7df6460472. Latest consensus state: current_term: 1 leader_uuid: "e35fd53df4bb4a2ea3857b7df6460472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } } }
I20260811 22:20:41.363801 9018 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:41.364203 9027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:41.365396 9017 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e35fd53df4bb4a2ea3857b7df6460472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35fd53df4bb4a2ea3857b7df6460472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42655 } } }
I20260811 22:20:41.365518 9017 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:41.367477 9027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:41.373327 9027 catalog_manager.cc:1347] Generated new cluster ID: 31da1ab730c844d0b43d8d04fc90fcec
I20260811 22:20:41.373416 9027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:41.380255 9027 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:41.381481 9027 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:41.387539 9027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35fd53df4bb4a2ea3857b7df6460472: Generated new TSK 0
I20260811 22:20:41.388470 9027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:41.400251 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 8930
W20260811 22:20:41.407472 8915 connection.cc:535] client connection to 127.0.0.1:42655 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42655): Connection reset by peer (error 104)
2026-08-11T22:20:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:41 dist-test-slave-77v9 krb5kdc[9040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:41 dist-test-slave-77v9 krb5kdc[9040](info): set up 2 sockets
Aug 11 22:20:41 dist-test-slave-77v9 krb5kdc[9040](info): commencing operation
krb5kdc: starting...
W20260811 22:20:42.537992 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:42 dist-test-slave-77v9 krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:42.681475 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:42.800544 9056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:42.800959 9056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:42.801062 9056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:42.805387 9056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:42.805522 9056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:42.805559 9056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:42.805598 9056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:42.805632 9056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:42.811319 9056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:42.812580 9056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:42.813750 9056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:42.822981 9062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:42.822981 9065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:42.822981 9063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:42.823372 9056 server_base.cc:1034] running on GCE node
I20260811 22:20:42.824052 9056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:42.825270 9056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:42.826453 9056 hybrid_clock.cc:648] HybridClock initialized: now 1786486842826418 us; error 30 us; skew 500 ppm
Aug 11 22:20:42 dist-test-slave-77v9 krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:42.829747 9056 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:42.830786 9056 webserver.cc:460] Webserver started at http://127.0.0.1:46523/ using document root <none> and password file <none>
I20260811 22:20:42.831394 9056 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:42.831498 9056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:42.831751 9056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:42.834040 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "f7ed7132dc8a4f42a824a0aee8825aca"
format_stamp: "Formatted at 2026-08-11 22:20:42 on dist-test-slave-77v9"
server_key: "4ce02439be6a2f8607a90ba85e80a280"
server_key_iv: "7406d82397663fdff61953eb2e5225f3"
server_key_version: "encryptionkey@0"
I20260811 22:20:42.834632 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "f7ed7132dc8a4f42a824a0aee8825aca"
format_stamp: "Formatted at 2026-08-11 22:20:42 on dist-test-slave-77v9"
server_key: "4ce02439be6a2f8607a90ba85e80a280"
server_key_iv: "7406d82397663fdff61953eb2e5225f3"
server_key_version: "encryptionkey@0"
I20260811 22:20:42.838790 9056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:42.841560 9072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:42.842913 9056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:42.843080 9056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "f7ed7132dc8a4f42a824a0aee8825aca"
format_stamp: "Formatted at 2026-08-11 22:20:42 on dist-test-slave-77v9"
server_key: "4ce02439be6a2f8607a90ba85e80a280"
server_key_iv: "7406d82397663fdff61953eb2e5225f3"
server_key_version: "encryptionkey@0"
I20260811 22:20:42.843214 9056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:42.848197 9056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:42.852061 9056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:42.852330 9056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:42.861444 9056 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46509
I20260811 22:20:42.861475 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46509 every 8 connection(s)
I20260811 22:20:42.862555 9056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:42.865623 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:42.870565 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9056
I20260811 22:20:42.870736 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:42.871060 18200 external_mini_cluster.cc:1428] Setting key 66ca0e13944005ac2d83218274aa88aa
I20260811 22:20:42.872236 9137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca: Bootstrap starting.
I20260811 22:20:42.874774 9137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:42.875669 9137 log.cc:824] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:42.876899 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:42.872735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:47959 (local address 127.0.0.1:46509)
0811 22:20:42.873241 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:42.873262 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:42.873299 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:20:42.873983 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:42.874000 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:42.874294 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:42.874648 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:42.875500 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:42.875504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:42.875515 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:42.875525 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:42.875577 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:42.875596 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:42.875713 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:42.875820 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:20:42.875988 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":141,"threads_started":1}
I20260811 22:20:42.878161 9137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca: No bootstrap required, opened a new log
I20260811 22:20:42.879792 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:42.882863 9137 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260811 22:20:42.883239 9137 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:42.883323 9137 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ed7132dc8a4f42a824a0aee8825aca, State: Initialized, Role: FOLLOWER
I20260811 22:20:42.883929 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:42.882065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:38457 (local address 127.0.0.1:46509)
0811 22:20:42.882188 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:42.882192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:42.882379 (+ 187us) server_negotiation.cc:407] Connection header received
0811 22:20:42.882551 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:42.882554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:42.882596 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:42.882681 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:42.883536 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:42.883538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:42.883540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:42.883551 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:42.883577 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:42.883587 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:42.883680 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:42.883755 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 22:20:42.883802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:20:42.884006 9137 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260811 22:20:42.884215 9137 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:42.884305 9137 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:42.884423 9137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:42.885705 9137 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260811 22:20:42.886118 9137 leader_election.cc:302] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7ed7132dc8a4f42a824a0aee8825aca; no voters:
I20260811 22:20:42.886502 9137 leader_election.cc:288] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:42.886615 9147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:42.886869 9147 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [term 1 LEADER]: Becoming Leader. State: Replica: f7ed7132dc8a4f42a824a0aee8825aca, State: Running, Role: LEADER
I20260811 22:20:42.887205 9147 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260811 22:20:42.888218 9137 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:42.889375 9149 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7ed7132dc8a4f42a824a0aee8825aca. Latest consensus state: current_term: 1 leader_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260811 22:20:42.889544 9149 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:42.889812 9148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ed7132dc8a4f42a824a0aee8825aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260811 22:20:42.889932 9148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:42.890013 9157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:42.893342 9157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:42.899554 9157 catalog_manager.cc:1347] Generated new cluster ID: 0636839d979d4d079954ba30ef77a9f6
I20260811 22:20:42.899694 9157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:42.911711 9157 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:42.912127 9157 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:42.918489 9157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7ed7132dc8a4f42a824a0aee8825aca: Generated new TSK 0
I20260811 22:20:42.919505 9157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:42.931154 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9056
2026-08-11T22:20:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:42 dist-test-slave-77v9 krb5kdc[9169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:42 dist-test-slave-77v9 krb5kdc[9169](info): set up 2 sockets
Aug 11 22:20:42 dist-test-slave-77v9 krb5kdc[9169](info): commencing operation
krb5kdc: starting...
W20260811 22:20:44.089309 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:44.237587 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:44.359361 9182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:44.359755 9182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:44.359853 9182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:44.364241 9182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:44.364377 9182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:44.364416 9182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:44.364460 9182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:44.364500 9182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:44.370543 9182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:44.371799 9182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:44.372929 9182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:44.381784 9187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:44.381949 9182 server_base.cc:1034] running on GCE node
W20260811 22:20:44.381784 9188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:44.381873 9190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:44.382740 9182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:44.383956 9182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:44.385182 9182 hybrid_clock.cc:648] HybridClock initialized: now 1786486844385154 us; error 41 us; skew 500 ppm
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:44.388401 9182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:44.389532 9182 webserver.cc:460] Webserver started at http://127.0.0.1:33055/ using document root <none> and password file <none>
I20260811 22:20:44.390136 9182 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:44.390233 9182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:44.390483 9182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:44.392640 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "eb92dfeb77054c4a937734ad31a15b42"
format_stamp: "Formatted at 2026-08-11 22:20:44 on dist-test-slave-77v9"
server_key: "8ff5e59575a6573e9d1ad9e4f52bb511"
server_key_iv: "459d05829ef37900ba002d8811884b96"
server_key_version: "encryptionkey@0"
I20260811 22:20:44.393339 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "eb92dfeb77054c4a937734ad31a15b42"
format_stamp: "Formatted at 2026-08-11 22:20:44 on dist-test-slave-77v9"
server_key: "8ff5e59575a6573e9d1ad9e4f52bb511"
server_key_iv: "459d05829ef37900ba002d8811884b96"
server_key_version: "encryptionkey@0"
I20260811 22:20:44.397570 9182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:20:44.400151 9199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:44.401620 9182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:44.401786 9182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "eb92dfeb77054c4a937734ad31a15b42"
format_stamp: "Formatted at 2026-08-11 22:20:44 on dist-test-slave-77v9"
server_key: "8ff5e59575a6573e9d1ad9e4f52bb511"
server_key_iv: "459d05829ef37900ba002d8811884b96"
server_key_version: "encryptionkey@0"
I20260811 22:20:44.401918 9182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:44.421554 9182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:44.425266 9182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:44.425560 9182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:44.433959 9182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35365
I20260811 22:20:44.433974 9258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35365 every 8 connection(s)
I20260811 22:20:44.435029 9182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:44.436137 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9182
I20260811 22:20:44.436280 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:44.436597 18200 external_mini_cluster.cc:1428] Setting key a5dfcfbf5f8c7d14b730f3cedf019f3b
I20260811 22:20:44.438627 9259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:44.445653 9259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42: Bootstrap starting.
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:20:44.448490 9259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:44.449466 9259 log.cc:824] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:44.452173 9259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42: No bootstrap required, opened a new log
I20260811 22:20:44.452752 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:44.438132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:45837 (local address 127.0.0.1:35365)
0811 22:20:44.438556 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:44.438575 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:44.438618 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:20:44.439525 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:44.439547 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:44.439966 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:44.440347 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:44.441114 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:44.442634 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:44.443535 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:44.444214 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:44.444389 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:44.447111 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:44.447118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:44.447136 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:44.447169 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:44.449284 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:44.449939 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:44.449949 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:44.449955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:44.450079 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:44.450487 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:44.450490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:44.450492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:44.451020 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:44.451230 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:44.451535 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:20:44.451770 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":123,"threads_started":1}
I20260811 22:20:44.456012 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:44.456497 9259 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 22:20:44.456744 9259 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:44.456833 9259 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb92dfeb77054c4a937734ad31a15b42, State: Initialized, Role: FOLLOWER
I20260811 22:20:44.457317 9259 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 22:20:44.457520 9259 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:44.457602 9259 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:44.457724 9259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:44.459021 9259 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 22:20:44.459410 9259 leader_election.cc:302] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb92dfeb77054c4a937734ad31a15b42; no voters:
I20260811 22:20:44.459745 9259 leader_election.cc:288] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:44.459851 9269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:44.460134 9269 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [term 1 LEADER]: Becoming Leader. State: Replica: eb92dfeb77054c4a937734ad31a15b42, State: Running, Role: LEADER
I20260811 22:20:44.460487 9269 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 22:20:44.461292 9259 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:44.463549 9270 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb92dfeb77054c4a937734ad31a15b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 22:20:44.463711 9270 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:44.464021 9271 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb92dfeb77054c4a937734ad31a15b42. Latest consensus state: current_term: 1 leader_uuid: "eb92dfeb77054c4a937734ad31a15b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb92dfeb77054c4a937734ad31a15b42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 22:20:44.464130 9271 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:44.464500 9280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:44.468128 9280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:44.472167 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:44.458397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:40325 (local address 127.0.0.1:35365)
0811 22:20:44.458541 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:44.458545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:44.461165 (+ 2620us) server_negotiation.cc:407] Connection header received
0811 22:20:44.461361 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:44.461367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:44.461427 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:44.461485 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:44.463046 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:44.464304 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:44.465645 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:44.465919 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:44.466130 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:44.467947 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:44.467951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:44.467953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:44.467982 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:44.469792 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:44.470481 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:44.470485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:44.470487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:44.470588 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:44.471041 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:44.471045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:44.471047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:44.471286 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:44.471452 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:44.471847 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 22:20:44.471986 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:20:44.475015 9280 catalog_manager.cc:1347] Generated new cluster ID: 402db8fb20394250977fa3844edc196c
I20260811 22:20:44.475132 9280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:44.489745 9280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:44.491111 9280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:44.498512 9280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb92dfeb77054c4a937734ad31a15b42: Generated new TSK 0
I20260811 22:20:44.499418 9280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:44.503605 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9182
W20260811 22:20:44.510540 9167 connection.cc:535] client connection to 127.0.0.1:35365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35365): Connection reset by peer (error 104)
2026-08-11T22:20:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9291](info): set up 2 sockets
Aug 11 22:20:44 dist-test-slave-77v9 krb5kdc[9291](info): commencing operation
krb5kdc: starting...
W20260811 22:20:45.701187 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:45 dist-test-slave-77v9 krb5kdc[9291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:45.844743 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:45.961351 9305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:45.961691 9305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:45.961771 9305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:45.966261 9305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:45.966430 9305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:45.966506 9305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:45.966538 9305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:45.966571 9305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:45.976001 9305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:45.977381 9305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:45.978487 9305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:45.987877 9311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:45.987937 9310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:45.987893 9313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:45.987963 9305 server_base.cc:1034] running on GCE node
I20260811 22:20:45.988708 9305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:45.989971 9305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:45.991146 9305 hybrid_clock.cc:648] HybridClock initialized: now 1786486845991125 us; error 36 us; skew 500 ppm
Aug 11 22:20:45 dist-test-slave-77v9 krb5kdc[9291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:45.999326 9305 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:46.000445 9305 webserver.cc:460] Webserver started at http://127.0.0.1:33549/ using document root <none> and password file <none>
I20260811 22:20:46.001199 9305 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:46.001272 9305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:46.001533 9305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:46.003742 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "d835d68e77754f40b839a36d43a25a06"
format_stamp: "Formatted at 2026-08-11 22:20:45 on dist-test-slave-77v9"
server_key: "577f8d461036cf22e83ef75a5354ff38"
server_key_iv: "aa77b1986a949e52d63e97b74d481885"
server_key_version: "encryptionkey@0"
I20260811 22:20:46.004357 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "d835d68e77754f40b839a36d43a25a06"
format_stamp: "Formatted at 2026-08-11 22:20:45 on dist-test-slave-77v9"
server_key: "577f8d461036cf22e83ef75a5354ff38"
server_key_iv: "aa77b1986a949e52d63e97b74d481885"
server_key_version: "encryptionkey@0"
I20260811 22:20:46.008569 9305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:20:46.011271 9320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:46.012691 9305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:20:46.012933 9305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "d835d68e77754f40b839a36d43a25a06"
format_stamp: "Formatted at 2026-08-11 22:20:45 on dist-test-slave-77v9"
server_key: "577f8d461036cf22e83ef75a5354ff38"
server_key_iv: "aa77b1986a949e52d63e97b74d481885"
server_key_version: "encryptionkey@0"
I20260811 22:20:46.013091 9305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:46.025575 9305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:46.029606 9305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:46.029867 9305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:46.038178 9305 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41745
I20260811 22:20:46.038209 9380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41745 every 8 connection(s)
I20260811 22:20:46.039340 9305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:46.042842 9381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:46.043125 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9305
I20260811 22:20:46.043299 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:46.043684 18200 external_mini_cluster.cc:1428] Setting key 7d55a76c3a1ce508c214dd70797ed512
I20260811 22:20:46.050128 9381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06: Bootstrap starting.
I20260811 22:20:46.052425 9381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:46.053648 9381 log.cc:824] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:46.053761 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:46.045346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:50293 (local address 127.0.0.1:41745)
0811 22:20:46.045847 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:46.045863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:46.045903 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:20:46.046626 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:46.046645 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:46.047011 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:46.047365 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:46.047818 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:46.048895 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:46.049827 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:46.050397 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:46.050573 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:46.051590 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:46.051594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:46.051607 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:46.052012 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:46.052075 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:46.052098 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:46.052243 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:46.052450 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:20:46.052596 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":167,"threads_started":1}
I20260811 22:20:46.055977 9381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06: No bootstrap required, opened a new log
I20260811 22:20:46.057075 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:46.060184 9381 raft_consensus.cc:348] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260811 22:20:46.060499 9381 raft_consensus.cc:374] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:46.060583 9381 raft_consensus.cc:729] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835d68e77754f40b839a36d43a25a06, State: Initialized, Role: FOLLOWER
I20260811 22:20:46.061089 9381 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260811 22:20:46.061244 9381 raft_consensus.cc:388] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:46.061335 9381 raft_consensus.cc:482] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:46.061436 9381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:46.062471 9381 raft_consensus.cc:504] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260811 22:20:46.062842 9381 leader_election.cc:302] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d835d68e77754f40b839a36d43a25a06; no voters:
I20260811 22:20:46.063263 9381 leader_election.cc:288] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:46.063349 9393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:46.063561 9393 raft_consensus.cc:686] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [term 1 LEADER]: Becoming Leader. State: Replica: d835d68e77754f40b839a36d43a25a06, State: Running, Role: LEADER
I20260811 22:20:46.063915 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:46.059264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:45929 (local address 127.0.0.1:41745)
0811 22:20:46.059394 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:46.059397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:46.059633 (+ 236us) server_negotiation.cc:407] Connection header received
0811 22:20:46.059797 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:46.059800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:46.059855 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:46.059952 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:46.060213 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:46.060961 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:46.061942 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:46.062212 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:46.062331 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:46.063259 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:46.063263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:46.063265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:46.063280 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:46.063337 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:46.063355 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:46.063457 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:46.063743 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:20:46.063786 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:20:46.063879 9393 consensus_queue.cc:237] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260811 22:20:46.064746 9381 sys_catalog.cc:564] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:46.065387 9393 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835d68e77754f40b839a36d43a25a06. Latest consensus state: current_term: 1 leader_uuid: "d835d68e77754f40b839a36d43a25a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } } }
I20260811 22:20:46.065534 9393 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:46.066052 9400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:46.065989 9394 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d835d68e77754f40b839a36d43a25a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835d68e77754f40b839a36d43a25a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } } }
I20260811 22:20:46.067991 9394 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:46.069458 9400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:46.075891 9400 catalog_manager.cc:1347] Generated new cluster ID: 768354c418144057b76717e8cbce4f61
I20260811 22:20:46.075994 9400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:46.100282 9400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:46.101740 9400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:46.110463 9400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d835d68e77754f40b839a36d43a25a06: Generated new TSK 0
I20260811 22:20:46.111366 9400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:46.125078 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9305
W20260811 22:20:46.133101 9289 connection.cc:535] client connection to 127.0.0.1:41745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41745): Connection reset by peer (error 104)
2026-08-11T22:20:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:46 dist-test-slave-77v9 krb5kdc[9416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:46 dist-test-slave-77v9 krb5kdc[9416](info): set up 2 sockets
Aug 11 22:20:46 dist-test-slave-77v9 krb5kdc[9416](info): commencing operation
krb5kdc: starting...
W20260811 22:20:47.309131 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:47 dist-test-slave-77v9 krb5kdc[9416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:20:47.453613 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:47.580796 9429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:47.581257 9429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:47.581344 9429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:47.586265 9429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:47.586362 9429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:47.586387 9429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:47.586409 9429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:47.586427 9429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:47.592975 9429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:47.594267 9429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:47.595408 9429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:47.604972 9434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:47.605015 9435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:47.605020 9437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:47.605170 9429 server_base.cc:1034] running on GCE node
I20260811 22:20:47.605880 9429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:47.607103 9429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:47.608281 9429 hybrid_clock.cc:648] HybridClock initialized: now 1786486847608250 us; error 46 us; skew 500 ppm
Aug 11 22:20:47 dist-test-slave-77v9 krb5kdc[9416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:47.611279 9429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:20:47.612334 9429 webserver.cc:460] Webserver started at http://127.0.0.1:35381/ using document root <none> and password file <none>
I20260811 22:20:47.612982 9429 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:47.613060 9429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:47.613246 9429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:47.615332 9429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "ec62547e9d104bdbad0a3bbf1e7230bb"
format_stamp: "Formatted at 2026-08-11 22:20:47 on dist-test-slave-77v9"
server_key: "1fae2c47155bef3c95840bc48b791973"
server_key_iv: "c520b9380d42199c811b2cb0d209adab"
server_key_version: "encryptionkey@0"
I20260811 22:20:47.615871 9429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "ec62547e9d104bdbad0a3bbf1e7230bb"
format_stamp: "Formatted at 2026-08-11 22:20:47 on dist-test-slave-77v9"
server_key: "1fae2c47155bef3c95840bc48b791973"
server_key_iv: "c520b9380d42199c811b2cb0d209adab"
server_key_version: "encryptionkey@0"
I20260811 22:20:47.619798 9429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:20:47.622339 9445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:47.623606 9429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:20:47.623726 9429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "ec62547e9d104bdbad0a3bbf1e7230bb"
format_stamp: "Formatted at 2026-08-11 22:20:47 on dist-test-slave-77v9"
server_key: "1fae2c47155bef3c95840bc48b791973"
server_key_iv: "c520b9380d42199c811b2cb0d209adab"
server_key_version: "encryptionkey@0"
I20260811 22:20:47.623823 9429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:47.628525 9429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:47.631505 9429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:47.631659 9429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:47.639380 9429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42765
I20260811 22:20:47.639408 9503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42765 every 8 connection(s)
I20260811 22:20:47.640375 9429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:47.642146 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9429
I20260811 22:20:47.642247 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:47.642557 18200 external_mini_cluster.cc:1428] Setting key 3584066d3f71c516bfae21eea1533359
I20260811 22:20:47.643450 9505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:47.648066 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:47.644025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:41681 (local address 127.0.0.1:42765)
0811 22:20:47.644496 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:47.644511 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:47.644547 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:20:47.645282 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:47.645300 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:47.645583 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:47.645922 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:47.646834 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:47.646839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:47.646850 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:47.646861 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:47.646917 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:47.646936 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:47.647100 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:47.647170 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:20:47.647336 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":158,"threads_started":1}
I20260811 22:20:47.650723 9505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: Bootstrap starting.
I20260811 22:20:47.651280 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:47.653393 9505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:47.654448 9505 log.cc:824] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:47.655603 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:47.653531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:35817 (local address 127.0.0.1:42765)
0811 22:20:47.653661 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:47.653667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:47.654008 (+ 341us) server_negotiation.cc:407] Connection header received
0811 22:20:47.654142 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:47.654144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:47.654174 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:47.654240 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:47.655105 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:47.655109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:47.655111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:47.655123 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:47.655150 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:47.655166 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:47.655266 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:47.655437 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:20:47.655486 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:20:47.657033 9505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: No bootstrap required, opened a new log
I20260811 22:20:47.660517 9505 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } }
I20260811 22:20:47.660785 9505 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:47.660884 9505 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec62547e9d104bdbad0a3bbf1e7230bb, State: Initialized, Role: FOLLOWER
I20260811 22:20:47.661386 9505 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } }
I20260811 22:20:47.661546 9505 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:47.661623 9505 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:47.661736 9505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:47.662698 9505 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } }
I20260811 22:20:47.663048 9505 leader_election.cc:302] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec62547e9d104bdbad0a3bbf1e7230bb; no voters:
I20260811 22:20:47.663372 9505 leader_election.cc:288] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:47.663604 9516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:47.663998 9516 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [term 1 LEADER]: Becoming Leader. State: Replica: ec62547e9d104bdbad0a3bbf1e7230bb, State: Running, Role: LEADER
I20260811 22:20:47.664333 9516 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } }
I20260811 22:20:47.664705 9505 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:47.666384 9517 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } } }
I20260811 22:20:47.666565 9517 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:47.666378 9518 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec62547e9d104bdbad0a3bbf1e7230bb. Latest consensus state: current_term: 1 leader_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec62547e9d104bdbad0a3bbf1e7230bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42765 } } }
I20260811 22:20:47.666858 9518 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:47.669703 9532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:47.669797 9532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:47.669870 9531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:47.670619 9531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:47.676899 9531 catalog_manager.cc:1347] Generated new cluster ID: 87db6694ce3b425da3c5d936b3ccb169
I20260811 22:20:47.677002 9531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:47.712285 9531 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:47.712670 9531 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:47.716526 9531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec62547e9d104bdbad0a3bbf1e7230bb: Generated new TSK 0
I20260811 22:20:47.717294 9531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:47.725600 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9429
2026-08-11T22:20:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:47 dist-test-slave-77v9 krb5kdc[9539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:47 dist-test-slave-77v9 krb5kdc[9539](info): set up 2 sockets
Aug 11 22:20:47 dist-test-slave-77v9 krb5kdc[9539](info): commencing operation
krb5kdc: starting...
W20260811 22:20:48.853097 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:48 dist-test-slave-77v9 krb5kdc[9539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:48.992295 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33449
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:33449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:49.107829 9552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:49.108155 9552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:49.108201 9552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:49.112490 9552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:49.112581 9552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:49.112600 9552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:49.112618 9552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:49.112633 9552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:49.118448 9552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:33449
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:33449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:49.119642 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:49.120731 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:49.129772 9560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:49.129786 9557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:49.129772 9558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:49.130194 9552 server_base.cc:1034] running on GCE node
I20260811 22:20:49.130868 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:49.132088 9552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:49.133287 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786486849133273 us; error 38 us; skew 500 ppm
Aug 11 22:20:49 dist-test-slave-77v9 krb5kdc[9539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:49.136590 9552 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:49.140085 9552 webserver.cc:460] Webserver started at http://127.17.198.62:36467/ using document root <none> and password file <none>
I20260811 22:20:49.140717 9552 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:49.140785 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:49.141085 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:49.143216 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "83902e3bf34b424e860b75d256d47865"
format_stamp: "Formatted at 2026-08-11 22:20:49 on dist-test-slave-77v9"
server_key: "5f915b2431e76eb42a7d69b5d45455d3"
server_key_iv: "0f12a783ab7f875712745f21b8cf6b03"
server_key_version: "encryptionkey@0"
I20260811 22:20:49.143764 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "83902e3bf34b424e860b75d256d47865"
format_stamp: "Formatted at 2026-08-11 22:20:49 on dist-test-slave-77v9"
server_key: "5f915b2431e76eb42a7d69b5d45455d3"
server_key_iv: "0f12a783ab7f875712745f21b8cf6b03"
server_key_version: "encryptionkey@0"
I20260811 22:20:49.147622 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:20:49.150286 9567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:49.151564 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:20:49.151681 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "83902e3bf34b424e860b75d256d47865"
format_stamp: "Formatted at 2026-08-11 22:20:49 on dist-test-slave-77v9"
server_key: "5f915b2431e76eb42a7d69b5d45455d3"
server_key_iv: "0f12a783ab7f875712745f21b8cf6b03"
server_key_version: "encryptionkey@0"
I20260811 22:20:49.151780 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:49.173095 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:49.176587 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:49.176759 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:49.184204 9552 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:33449
I20260811 22:20:49.184255 9627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:33449 every 8 connection(s)
I20260811 22:20:49.185220 9552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:49.187975 9629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:49.190882 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9552
I20260811 22:20:49.191039 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:49.191354 18200 external_mini_cluster.cc:1428] Setting key 75bb710e1bcd449e0057439ffe7e7ff9
I20260811 22:20:49.194470 9629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: Bootstrap starting.
I20260811 22:20:49.197146 9629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:49.198102 9629 log.cc:824] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:49.200587 9629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: No bootstrap required, opened a new log
Aug 11 22:20:49 dist-test-slave-77v9 krb5kdc[9539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:49.203671 9629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } }
I20260811 22:20:49.203920 9629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:49.203961 9629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83902e3bf34b424e860b75d256d47865, State: Initialized, Role: FOLLOWER
I20260811 22:20:49.204464 9629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } }
I20260811 22:20:49.204591 9629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:49.204674 9629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:49.204789 9629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:49.205778 9629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } }
I20260811 22:20:49.206189 9629 leader_election.cc:302] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83902e3bf34b424e860b75d256d47865; no voters:
I20260811 22:20:49.206593 9629 leader_election.cc:288] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:49.206712 9634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:49.206967 9634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [term 1 LEADER]: Becoming Leader. State: Replica: 83902e3bf34b424e860b75d256d47865, State: Running, Role: LEADER
I20260811 22:20:49.207312 9634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } }
I20260811 22:20:49.208032 9629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:49.208614 9634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83902e3bf34b424e860b75d256d47865. Latest consensus state: current_term: 1 leader_uuid: "83902e3bf34b424e860b75d256d47865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } } }
I20260811 22:20:49.208739 9634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:49.209584 9636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83902e3bf34b424e860b75d256d47865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83902e3bf34b424e860b75d256d47865" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 33449 } } }
I20260811 22:20:49.209767 9636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:49.211467 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:49.192940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34577 (local address 127.17.198.62:33449)
0811 22:20:49.193406 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:49.193422 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:49.193480 (+ 58us) server_negotiation.cc:407] Connection header received
0811 22:20:49.194113 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:49.194129 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:49.194477 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:49.194800 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:49.195707 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:49.196718 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:49.197967 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:49.198554 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:49.198736 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:49.201719 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:49.201725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:49.201738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:49.201768 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:49.204285 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:49.204973 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:49.204978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:49.204983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:49.205098 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:49.205477 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:49.205479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:49.205481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:49.205878 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:49.206099 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:49.206491 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 22:20:49.209434 (+ 2943us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":144,"threads_started":1}
W20260811 22:20:49.214432 9653 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:49.214541 9653 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:49.214609 9641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:49.215291 9641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:49.216912 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:49.221900 9641 catalog_manager.cc:1347] Generated new cluster ID: 7bac422661bc49ee858ad86503d3aafc
I20260811 22:20:49.221983 9641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:49.228744 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:49.219706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47433 (local address 127.17.198.62:33449)
0811 22:20:49.219843 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:49.219848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:49.220120 (+ 272us) server_negotiation.cc:407] Connection header received
0811 22:20:49.220363 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:49.220366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:49.220425 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:49.220479 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:49.221578 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:49.222325 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:49.223208 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:49.223469 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:49.223592 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:49.225212 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:49.225216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:49.225221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:49.225252 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:49.226778 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:49.227348 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:49.227351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:49.227352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:49.227428 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:49.227780 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:49.227783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:49.227785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:49.227991 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:49.228170 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:49.228447 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:20:49.228551 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:20:49.241510 9641 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:49.242704 9641 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:49.250204 9641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83902e3bf34b424e860b75d256d47865: Generated new TSK 0
I20260811 22:20:49.251063 9641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:49.264134 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9552
W20260811 22:20:49.273500 9536 connection.cc:535] client connection to 127.17.198.62:33449 recv error: Network error: failed to read from TLS socket (remote: 127.17.198.62:33449): Connection reset by peer (error 104)
2026-08-11T22:20:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:49 dist-test-slave-77v9 krb5kdc[9665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:49 dist-test-slave-77v9 krb5kdc[9665](info): set up 2 sockets
Aug 11 22:20:49 dist-test-slave-77v9 krb5kdc[9665](info): commencing operation
krb5kdc: starting...
W20260811 22:20:50.424376 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:50 dist-test-slave-77v9 krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:50.570573 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40385
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:40385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:50.698971 9679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:50.699378 9679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:50.699476 9679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:50.703797 9679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:50.703948 9679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:50.703989 9679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:50.704028 9679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:50.704062 9679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:50.709985 9679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:40385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:50.711216 9679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:50.712311 9679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:20:50.721724 9679 server_base.cc:1034] running on GCE node
W20260811 22:20:50.721710 9684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:50.721732 9686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:50.721745 9688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:50.722705 9679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:50.724000 9679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:50.725215 9679 hybrid_clock.cc:648] HybridClock initialized: now 1786486850725150 us; error 82 us; skew 500 ppm
Aug 11 22:20:50 dist-test-slave-77v9 krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:50.728607 9679 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:50.729839 9679 webserver.cc:460] Webserver started at http://127.17.198.62:42327/ using document root <none> and password file <none>
I20260811 22:20:50.730481 9679 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:50.730587 9679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:50.730852 9679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:50.733304 9679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c4751229547e4a9c89aa9d582106d0b7"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-77v9"
server_key: "f21d91df9a37cdfcb0add7f0e174c652"
server_key_iv: "35cc5d79c3d4d510507451c3c089a603"
server_key_version: "encryptionkey@0"
I20260811 22:20:50.733950 9679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c4751229547e4a9c89aa9d582106d0b7"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-77v9"
server_key: "f21d91df9a37cdfcb0add7f0e174c652"
server_key_iv: "35cc5d79c3d4d510507451c3c089a603"
server_key_version: "encryptionkey@0"
I20260811 22:20:50.738204 9679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:50.741256 9695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:50.742939 9679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:20:50.743134 9679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c4751229547e4a9c89aa9d582106d0b7"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-77v9"
server_key: "f21d91df9a37cdfcb0add7f0e174c652"
server_key_iv: "35cc5d79c3d4d510507451c3c089a603"
server_key_version: "encryptionkey@0"
I20260811 22:20:50.743286 9679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:50.759021 9679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:50.762845 9679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:50.763070 9679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:50.771142 9679 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:40385
I20260811 22:20:50.771170 9751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:40385 every 8 connection(s)
I20260811 22:20:50.772222 9679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:50.775475 9752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:50.779526 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9679
I20260811 22:20:50.779719 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:50.780059 18200 external_mini_cluster.cc:1428] Setting key d837bbf5b01de7d69a87fddacb5eec78
I20260811 22:20:50.782603 9752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: Bootstrap starting.
I20260811 22:20:50.785179 9752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:50.786072 9752 log.cc:824] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:50.788643 9752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: No bootstrap required, opened a new log
I20260811 22:20:50.790114 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:50.781796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42805 (local address 127.17.198.62:40385)
0811 22:20:50.782309 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:50.782325 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:50.782363 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:20:50.783142 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:50.783161 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:50.783525 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:50.783877 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:50.784316 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.785447 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.786530 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.787047 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.787233 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:50.788269 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:50.788274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:50.788289 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:50.788704 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:50.788793 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:50.788859 (+ 66us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:50.789009 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:50.789213 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:20:50.789369 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":167,"threads_started":1}
I20260811 22:20:50.793570 9752 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } }
I20260811 22:20:50.793951 9752 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:50.794034 9752 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4751229547e4a9c89aa9d582106d0b7, State: Initialized, Role: FOLLOWER
I20260811 22:20:50.794073 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:50.794565 9752 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } }
I20260811 22:20:50.794766 9752 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:50.794849 9752 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:50.794970 9752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:50.796485 9752 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } }
I20260811 22:20:50.796991 9752 leader_election.cc:302] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4751229547e4a9c89aa9d582106d0b7; no voters:
I20260811 22:20:50.797406 9752 leader_election.cc:288] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:50.797544 9762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:50.797765 9762 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [term 1 LEADER]: Becoming Leader. State: Replica: c4751229547e4a9c89aa9d582106d0b7, State: Running, Role: LEADER
I20260811 22:20:50.798123 9762 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } }
I20260811 22:20:50.798947 9752 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:50.799487 9762 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4751229547e4a9c89aa9d582106d0b7. Latest consensus state: current_term: 1 leader_uuid: "c4751229547e4a9c89aa9d582106d0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } } }
I20260811 22:20:50.799613 9762 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:50.799927 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4751229547e4a9c89aa9d582106d0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4751229547e4a9c89aa9d582106d0b7" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40385 } } }
I20260811 22:20:50.800040 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:50.802125 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:50.796711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35057 (local address 127.17.198.62:40385)
0811 22:20:50.796889 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:50.796895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:50.797166 (+ 271us) server_negotiation.cc:407] Connection header received
0811 22:20:50.797331 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:50.797335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:50.797391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:50.797488 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:50.797871 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.798582 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.799714 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.800092 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.800178 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:50.801495 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:50.801499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:50.801503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:50.801518 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:50.801551 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:50.801569 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:50.801644 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:50.801917 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 22:20:50.801958 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 22:20:50.804641 9781 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:50.804725 9781 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:50.804855 9769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:50.805735 9769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:50.812105 9769 catalog_manager.cc:1347] Generated new cluster ID: d33d07217f25447b8aa9c5ab9f51be1d
I20260811 22:20:50.812215 9769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:50.834225 9769 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:50.835666 9769 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:50.849643 9769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4751229547e4a9c89aa9d582106d0b7: Generated new TSK 0
I20260811 22:20:50.850617 9769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:50.861294 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9679
W20260811 22:20:50.870772 9663 connection.cc:535] client connection to 127.17.198.62:40385 recv error: Network error: failed to read from TLS socket (remote: 127.17.198.62:40385): Connection reset by peer (error 104)
2026-08-11T22:20:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:50 dist-test-slave-77v9 krb5kdc[9787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:50 dist-test-slave-77v9 krb5kdc[9787](info): set up 2 sockets
Aug 11 22:20:50 dist-test-slave-77v9 krb5kdc[9787](info): commencing operation
krb5kdc: starting...
W20260811 22:20:52.020712 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:52 dist-test-slave-77v9 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:52.169745 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45059
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:40487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:45059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:52.296119 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:52.296500 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:52.296597 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:52.300840 9803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:52.300972 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:52.301009 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:52.301049 9803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:52.301101 9803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:52.306885 9803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:40487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:45059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:52.308147 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:52.309253 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:52.318467 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:52.318506 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:52.318501 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:52.318550 9803 server_base.cc:1034] running on GCE node
I20260811 22:20:52.319372 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:52.320616 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:52.321825 9803 hybrid_clock.cc:648] HybridClock initialized: now 1786486852321810 us; error 39 us; skew 500 ppm
Aug 11 22:20:52 dist-test-slave-77v9 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486852, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:52.331704 9803 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:52.332715 9803 webserver.cc:460] Webserver started at http://127.17.198.62:40559/ using document root <none> and password file <none>
I20260811 22:20:52.333375 9803 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:52.333451 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:52.333633 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:52.335851 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "6a6bb84fe7134486ac61e4621abd6c7a"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-77v9"
server_key: "42569a57901aca6bc1431c51d9cd1640"
server_key_iv: "acaced43880945d4a420e1923d6aaf81"
server_key_version: "encryptionkey@0"
I20260811 22:20:52.336411 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "6a6bb84fe7134486ac61e4621abd6c7a"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-77v9"
server_key: "42569a57901aca6bc1431c51d9cd1640"
server_key_iv: "acaced43880945d4a420e1923d6aaf81"
server_key_version: "encryptionkey@0"
I20260811 22:20:52.340445 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:20:52.343217 9819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:52.344571 9803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:20:52.344686 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "6a6bb84fe7134486ac61e4621abd6c7a"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-77v9"
server_key: "42569a57901aca6bc1431c51d9cd1640"
server_key_iv: "acaced43880945d4a420e1923d6aaf81"
server_key_version: "encryptionkey@0"
I20260811 22:20:52.344789 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:52.349687 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:52.353233 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:52.353402 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:52.361377 9803 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:45059
I20260811 22:20:52.361402 9877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:45059 every 8 connection(s)
I20260811 22:20:52.362430 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:52.365419 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:52.368803 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9803
I20260811 22:20:52.368997 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:52.369315 18200 external_mini_cluster.cc:1428] Setting key 687cb07dba30e041eb69367bf3e73c6a
I20260811 22:20:52.371893 9878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: Bootstrap starting.
I20260811 22:20:52.374459 9878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:52.375094 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.370958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39201 (local address 127.17.198.62:45059)
0811 22:20:52.371437 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.371458 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.371497 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:20:52.372223 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.372242 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.372533 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.372895 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.373766 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.373771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.373782 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:52.373793 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.373850 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:52.373869 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.374008 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.374113 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.374280 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":155,"threads_started":1}
I20260811 22:20:52.375291 9878 log.cc:824] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:52.377614 9878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: No bootstrap required, opened a new log
I20260811 22:20:52.378201 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:52.381796 9878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } }
I20260811 22:20:52.382124 9878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:52.382208 9878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6bb84fe7134486ac61e4621abd6c7a, State: Initialized, Role: FOLLOWER
I20260811 22:20:52.382570 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.380554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35619 (local address 127.17.198.62:45059)
0811 22:20:52.380716 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.380722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.380917 (+ 195us) server_negotiation.cc:407] Connection header received
0811 22:20:52.381062 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.381064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.381097 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.381177 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.382042 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.382045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.382047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:52.382058 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.382084 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:52.382115 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.382238 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.382395 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.382442 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:20:52.382783 9878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } }
I20260811 22:20:52.382941 9878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:52.383011 9878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:52.383116 9878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:52.384177 9878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } }
I20260811 22:20:52.384488 9878 leader_election.cc:302] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a6bb84fe7134486ac61e4621abd6c7a; no voters:
I20260811 22:20:52.384752 9878 leader_election.cc:288] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:52.384897 9890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:52.385191 9890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [term 1 LEADER]: Becoming Leader. State: Replica: 6a6bb84fe7134486ac61e4621abd6c7a, State: Running, Role: LEADER
I20260811 22:20:52.385583 9890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } }
I20260811 22:20:52.386099 9878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:52.387471 9892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a6bb84fe7134486ac61e4621abd6c7a. Latest consensus state: current_term: 1 leader_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } } }
I20260811 22:20:52.387610 9892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:52.388041 9891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bb84fe7134486ac61e4621abd6c7a" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45059 } } }
I20260811 22:20:52.388139 9891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a [sys.catalog]: This master's current role is: LEADER
W20260811 22:20:52.391147 9905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:52.391232 9905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:52.391315 9900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:52.391995 9900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:52.397814 9900 catalog_manager.cc:1347] Generated new cluster ID: a248609525254d71bea96665f5a82d83
I20260811 22:20:52.397903 9900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:52.416169 9900 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:52.416577 9900 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:52.425051 9900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a6bb84fe7134486ac61e4621abd6c7a: Generated new TSK 0
I20260811 22:20:52.425859 9900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:52.431157 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9803
2026-08-11T22:20:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:52 dist-test-slave-77v9 krb5kdc[9911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:52 dist-test-slave-77v9 krb5kdc[9911](info): set up 2 sockets
Aug 11 22:20:52 dist-test-slave-77v9 krb5kdc[9911](info): commencing operation
krb5kdc: starting...
W20260811 22:20:53.573812 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:53 dist-test-slave-77v9 krb5kdc[9911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:53.720250 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45409
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:45409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:53.842063 9926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:53.842443 9926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:53.842523 9926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:53.846908 9926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:53.847028 9926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:53.847065 9926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:53.847107 9926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:53.847146 9926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:53.853003 9926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:45409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:53.854270 9926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:53.855383 9926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:53.864943 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:53.865029 9926 server_base.cc:1034] running on GCE node
W20260811 22:20:53.864955 9935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:53.864974 9932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:53.865775 9926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:53.867022 9926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:53.868245 9926 hybrid_clock.cc:648] HybridClock initialized: now 1786486853868204 us; error 47 us; skew 500 ppm
Aug 11 22:20:53 dist-test-slave-77v9 krb5kdc[9911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486853, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:53.871328 9926 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:53.872469 9926 webserver.cc:460] Webserver started at http://127.17.198.62:39673/ using document root <none> and password file <none>
I20260811 22:20:53.873181 9926 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:53.873281 9926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:53.873524 9926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:53.875725 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "c2c940dfc1b94fb29d47f4d82189ae17"
format_stamp: "Formatted at 2026-08-11 22:20:53 on dist-test-slave-77v9"
server_key: "97dcb19758239280881e658dd0a57b1e"
server_key_iv: "192e4695e64b08dcfd965bb979078806"
server_key_version: "encryptionkey@0"
I20260811 22:20:53.876328 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "c2c940dfc1b94fb29d47f4d82189ae17"
format_stamp: "Formatted at 2026-08-11 22:20:53 on dist-test-slave-77v9"
server_key: "97dcb19758239280881e658dd0a57b1e"
server_key_iv: "192e4695e64b08dcfd965bb979078806"
server_key_version: "encryptionkey@0"
I20260811 22:20:53.880286 9926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:20:53.882977 9943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:53.884353 9926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:20:53.884513 9926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "c2c940dfc1b94fb29d47f4d82189ae17"
format_stamp: "Formatted at 2026-08-11 22:20:53 on dist-test-slave-77v9"
server_key: "97dcb19758239280881e658dd0a57b1e"
server_key_iv: "192e4695e64b08dcfd965bb979078806"
server_key_version: "encryptionkey@0"
I20260811 22:20:53.884649 9926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:53.909580 9926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:53.913111 9926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:53.913362 9926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:53.921324 9926 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:45409
I20260811 22:20:53.921351 10001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:45409 every 8 connection(s)
I20260811 22:20:53.922436 9926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:53.925587 10002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:53.928772 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 9926
I20260811 22:20:53.929011 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:53.929342 18200 external_mini_cluster.cc:1428] Setting key bdf69bbd7209b8aaa2344fa7fa8f5134
I20260811 22:20:53.932559 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17: Bootstrap starting.
I20260811 22:20:53.934996 10002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:53.935801 10002 log.cc:824] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:53.938086 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17: No bootstrap required, opened a new log
Aug 11 22:20:53 dist-test-slave-77v9 krb5kdc[9911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:53.941956 10002 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } }
I20260811 22:20:53.942214 10002 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:53.942284 10002 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c940dfc1b94fb29d47f4d82189ae17, State: Initialized, Role: FOLLOWER
I20260811 22:20:53.942770 10002 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } }
I20260811 22:20:53.942934 10002 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:53.943025 10002 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:53.943151 10002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:53.944132 10002 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } }
I20260811 22:20:53.944514 10002 leader_election.cc:302] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2c940dfc1b94fb29d47f4d82189ae17; no voters:
I20260811 22:20:53.944926 10002 leader_election.cc:288] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:53.945041 10008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:53.945256 10008 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [term 1 LEADER]: Becoming Leader. State: Replica: c2c940dfc1b94fb29d47f4d82189ae17, State: Running, Role: LEADER
I20260811 22:20:53.945647 10008 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } }
I20260811 22:20:53.946444 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.930995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50863 (local address 127.17.198.62:45409)
0811 22:20:53.931580 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:53.931597 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:53.931638 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:20:53.932375 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:53.932396 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:53.932912 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:53.933280 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:53.934190 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.935289 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.936249 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.936742 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.936965 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:53.939918 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:53.939924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:53.939938 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:53.939967 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:53.942763 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.943361 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.943367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.943373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.943508 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.943817 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.943822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.943823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.944252 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:53.944488 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:53.944794 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:20:53.945441 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":180,"threads_started":1}
I20260811 22:20:53.946456 10002 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:53.947408 10010 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } } }
I20260811 22:20:53.947810 10010 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:53.947711 10009 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2c940dfc1b94fb29d47f4d82189ae17. Latest consensus state: current_term: 1 leader_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c940dfc1b94fb29d47f4d82189ae17" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 45409 } } }
I20260811 22:20:53.948170 10009 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:53.948259 10018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:53.950079 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:53.951383 10018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:53.958225 10018 catalog_manager.cc:1347] Generated new cluster ID: 690664ae3d2e4245a9581d37249f98ff
I20260811 22:20:53.958315 10018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:53.964082 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.952934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40729 (local address 127.17.198.62:45409)
0811 22:20:53.953076 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:53.953081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:53.953097 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:53.953145 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:53.953148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:53.953195 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:53.953290 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:53.954524 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.955548 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.956986 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.957468 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.957580 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:53.960233 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:53.960237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:53.960240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:53.960292 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:53.962158 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.962739 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.962742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.962744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.962846 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.963148 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.963151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.963153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.963365 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:53.963490 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:53.963784 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:20:53.963905 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:20:53.978600 10018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:53.979947 10018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:53.985270 10018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2c940dfc1b94fb29d47f4d82189ae17: Generated new TSK 0
I20260811 22:20:53.986135 10018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:53.992019 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 9926
2026-08-11T22:20:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:54 dist-test-slave-77v9 krb5kdc[10036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:54 dist-test-slave-77v9 krb5kdc[10036](info): set up 2 sockets
Aug 11 22:20:54 dist-test-slave-77v9 krb5kdc[10036](info): commencing operation
krb5kdc: starting...
W20260811 22:20:55.153306 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:55 dist-test-slave-77v9 krb5kdc[10036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:55.304625 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43405
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:43405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:55.425510 10050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:55.425874 10050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:55.425963 10050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:55.430354 10050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:55.430461 10050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:55.430503 10050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:55.430544 10050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:55.430576 10050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:55.438218 10050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:43405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:43405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:55.439555 10050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:55.440641 10050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:55.450233 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:55.450235 10056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:55.450233 10057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:55.450595 10050 server_base.cc:1034] running on GCE node
I20260811 22:20:55.451385 10050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:55.452728 10050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:55.453941 10050 hybrid_clock.cc:648] HybridClock initialized: now 1786486855453911 us; error 35 us; skew 500 ppm
Aug 11 22:20:55 dist-test-slave-77v9 krb5kdc[10036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486855, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:55.456980 10050 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:55.458037 10050 webserver.cc:460] Webserver started at http://127.17.198.62:42111/ using document root <none> and password file <none>
I20260811 22:20:55.458642 10050 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:55.458740 10050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:55.459000 10050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:55.461241 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "538b9c41734e40e9ac241e0872263e97"
format_stamp: "Formatted at 2026-08-11 22:20:55 on dist-test-slave-77v9"
server_key: "a7f04a518c1e8757a3a741215fa61288"
server_key_iv: "4409a5412a91a46d6eb25389713df851"
server_key_version: "encryptionkey@0"
I20260811 22:20:55.461869 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "538b9c41734e40e9ac241e0872263e97"
format_stamp: "Formatted at 2026-08-11 22:20:55 on dist-test-slave-77v9"
server_key: "a7f04a518c1e8757a3a741215fa61288"
server_key_iv: "4409a5412a91a46d6eb25389713df851"
server_key_version: "encryptionkey@0"
I20260811 22:20:55.466094 10050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:20:55.469026 10068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:55.470408 10050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:55.470582 10050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "538b9c41734e40e9ac241e0872263e97"
format_stamp: "Formatted at 2026-08-11 22:20:55 on dist-test-slave-77v9"
server_key: "a7f04a518c1e8757a3a741215fa61288"
server_key_iv: "4409a5412a91a46d6eb25389713df851"
server_key_version: "encryptionkey@0"
I20260811 22:20:55.470727 10050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:55.484697 10050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:55.488384 10050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:55.488677 10050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:55.496815 10050 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:43405
I20260811 22:20:55.496860 10129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:43405 every 8 connection(s)
I20260811 22:20:55.497875 10050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:55.500694 10130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:55.503593 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10050
I20260811 22:20:55.503748 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:55.504077 18200 external_mini_cluster.cc:1428] Setting key 8dda607ba634ad7d898d6b0b758c38a2
I20260811 22:20:55.507025 10130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97: Bootstrap starting.
I20260811 22:20:55.509402 10130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:55.510175 10130 log.cc:824] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:55.512216 10130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97: No bootstrap required, opened a new log
I20260811 22:20:55.513562 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:55.505601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51161 (local address 127.17.198.62:43405)
0811 22:20:55.506091 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:55.506105 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:55.506142 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:20:55.506808 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:55.506826 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:55.507204 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:55.507566 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:55.508020 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:55.509105 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:55.510065 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:55.510603 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:55.510776 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:55.511688 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:55.511693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:55.511707 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:55.512097 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:55.512156 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:55.512180 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:55.512317 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:55.512482 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:20:55.512697 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":166,"threads_started":1}
I20260811 22:20:55.516718 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:55.516638 10130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } }
I20260811 22:20:55.516980 10130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:55.517061 10130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 538b9c41734e40e9ac241e0872263e97, State: Initialized, Role: FOLLOWER
I20260811 22:20:55.517477 10130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } }
I20260811 22:20:55.517632 10130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:55.517720 10130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:55.517822 10130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:55.518954 10130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } }
I20260811 22:20:55.519400 10130 leader_election.cc:302] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 538b9c41734e40e9ac241e0872263e97; no voters:
I20260811 22:20:55.519802 10130 leader_election.cc:288] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:55.519968 10141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:55.520202 10141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [term 1 LEADER]: Becoming Leader. State: Replica: 538b9c41734e40e9ac241e0872263e97, State: Running, Role: LEADER
I20260811 22:20:55.520574 10141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } }
I20260811 22:20:55.521343 10130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:55.522392 10142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "538b9c41734e40e9ac241e0872263e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } } }
I20260811 22:20:55.522557 10142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:55.522861 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 538b9c41734e40e9ac241e0872263e97. Latest consensus state: current_term: 1 leader_uuid: "538b9c41734e40e9ac241e0872263e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "538b9c41734e40e9ac241e0872263e97" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 43405 } } }
I20260811 22:20:55.522974 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:55.523269 10154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:55.524760 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:55.519207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47295 (local address 127.17.198.62:43405)
0811 22:20:55.519355 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:55.519360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:55.519612 (+ 252us) server_negotiation.cc:407] Connection header received
0811 22:20:55.519798 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:55.519802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:55.519861 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:55.519949 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:55.520444 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:55.521237 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:55.522521 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:55.522900 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:55.523009 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:55.524098 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:55.524104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:55.524107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:55.524149 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:55.524180 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:55.524194 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:55.524267 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:55.524441 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:20:55.524547 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:20:55.526726 10154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:55.533188 10154 catalog_manager.cc:1347] Generated new cluster ID: d321d8c259c54c099eed9111c12d437b
I20260811 22:20:55.533296 10154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:55.553969 10154 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:55.555249 10154 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:55.572185 10154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 538b9c41734e40e9ac241e0872263e97: Generated new TSK 0
I20260811 22:20:55.573163 10154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:55.581063 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10050
2026-08-11T22:20:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:55 dist-test-slave-77v9 krb5kdc[10167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:55 dist-test-slave-77v9 krb5kdc[10167](info): set up 2 sockets
Aug 11 22:20:55 dist-test-slave-77v9 krb5kdc[10167](info): commencing operation
krb5kdc: starting...
W20260811 22:20:56.735647 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:56 dist-test-slave-77v9 krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:56.886270 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37577
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:37577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:57.006044 10182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:57.006433 10182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:57.006553 10182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:57.010813 10182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:57.010937 10182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:57.010977 10182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:57.011018 10182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:57.011054 10182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:57.016736 10182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:37577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:37577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:57.018060 10182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:57.019196 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:57.028448 10189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:57.028434 10191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:57.028484 10188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:57.028677 10182 server_base.cc:1034] running on GCE node
I20260811 22:20:57.029470 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:57.030668 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:57.031872 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786486857031841 us; error 52 us; skew 500 ppm
Aug 11 22:20:57 dist-test-slave-77v9 krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:57.034919 10182 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:57.035980 10182 webserver.cc:460] Webserver started at http://127.17.198.62:41441/ using document root <none> and password file <none>
I20260811 22:20:57.036599 10182 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:57.036666 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:57.036922 10182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:57.039111 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "1dd58088c6b746a889caad653d33bc0f"
format_stamp: "Formatted at 2026-08-11 22:20:57 on dist-test-slave-77v9"
server_key: "7106179f800636ec43d15829fe2d250d"
server_key_iv: "6ae197e5c8ae7dc092397d0b6dc83dcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:57.039701 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "1dd58088c6b746a889caad653d33bc0f"
format_stamp: "Formatted at 2026-08-11 22:20:57 on dist-test-slave-77v9"
server_key: "7106179f800636ec43d15829fe2d250d"
server_key_iv: "6ae197e5c8ae7dc092397d0b6dc83dcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:57.043975 10182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:20:57.046856 10198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:57.048434 10182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:20:57.048568 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "1dd58088c6b746a889caad653d33bc0f"
format_stamp: "Formatted at 2026-08-11 22:20:57 on dist-test-slave-77v9"
server_key: "7106179f800636ec43d15829fe2d250d"
server_key_iv: "6ae197e5c8ae7dc092397d0b6dc83dcc"
server_key_version: "encryptionkey@0"
I20260811 22:20:57.048686 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:57.053787 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:57.057454 10182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:57.057628 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:57.065567 10182 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:37577
I20260811 22:20:57.065596 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:37577 every 8 connection(s)
I20260811 22:20:57.066675 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:57.069741 10259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:57.074734 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10182
I20260811 22:20:57.074862 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:57.075099 18200 external_mini_cluster.cc:1428] Setting key 5b2c3db5aa2c1cc669fb7203d4070f27
I20260811 22:20:57.076570 10259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f: Bootstrap starting.
I20260811 22:20:57.079159 10259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:57.080056 10259 log.cc:824] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:57.080957 10262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:57.076698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59337 (local address 127.17.198.62:37577)
0811 22:20:57.077236 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:57.077252 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:57.077291 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:20:57.078023 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:57.078043 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:57.078349 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:57.078721 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:57.079627 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:57.079632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:57.079644 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:57.079655 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:57.079710 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:57.079729 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:57.079852 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:57.079954 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:20:57.080126 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":165,"threads_started":1}
I20260811 22:20:57.082355 10259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f: No bootstrap required, opened a new log
I20260811 22:20:57.083931 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:57.087486 10259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } }
I20260811 22:20:57.087845 10259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:57.087931 10259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd58088c6b746a889caad653d33bc0f, State: Initialized, Role: FOLLOWER
I20260811 22:20:57.088375 10262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:57.086459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53681 (local address 127.17.198.62:37577)
0811 22:20:57.086614 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:57.086621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:57.086788 (+ 167us) server_negotiation.cc:407] Connection header received
0811 22:20:57.086926 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:57.086929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:57.086962 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:57.087027 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:57.087803 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:57.087829 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:57.087834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:20:57.087848 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:57.087887 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:20:57.087901 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:57.088038 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:57.088186 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 22:20:57.088231 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:20:57.088444 10259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } }
I20260811 22:20:57.088626 10259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:57.088694 10259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:57.088794 10259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:57.089936 10259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } }
I20260811 22:20:57.090245 10259 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd58088c6b746a889caad653d33bc0f; no voters:
I20260811 22:20:57.090517 10259 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:57.090663 10269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:57.090919 10269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [term 1 LEADER]: Becoming Leader. State: Replica: 1dd58088c6b746a889caad653d33bc0f, State: Running, Role: LEADER
I20260811 22:20:57.091300 10269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } }
I20260811 22:20:57.091837 10259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:57.093245 10270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd58088c6b746a889caad653d33bc0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } } }
I20260811 22:20:57.093384 10270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:57.093809 10280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:57.093904 10271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd58088c6b746a889caad653d33bc0f. Latest consensus state: current_term: 1 leader_uuid: "1dd58088c6b746a889caad653d33bc0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd58088c6b746a889caad653d33bc0f" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 37577 } } }
I20260811 22:20:57.094074 10271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:57.096722 10280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:57.102098 10280 catalog_manager.cc:1347] Generated new cluster ID: b799f6fc2f9748e09db5c5d3dbc1fcdf
I20260811 22:20:57.102181 10280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:57.120496 10280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:57.120944 10280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:57.130448 10280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd58088c6b746a889caad653d33bc0f: Generated new TSK 0
I20260811 22:20:57.131299 10280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:20:57.137606 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10182
2026-08-11T22:20:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1555 ms)
[----------] 18 tests from AuthTokenIssuingTest (28420 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:57 dist-test-slave-77v9 krb5kdc[10294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:57 dist-test-slave-77v9 krb5kdc[10294](info): set up 2 sockets
Aug 11 22:20:57 dist-test-slave-77v9 krb5kdc[10294](info): commencing operation
krb5kdc: starting...
W20260811 22:20:58.285341 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:20:58.432878 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40783
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:58.556578 10311 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:20:58.556932 10311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:58.556957 10311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:58.557041 10311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:58.561297 10311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:58.561373 10311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:58.561393 10311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:58.561410 10311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:58.561425 10311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:58.567056 10311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:58.568289 10311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:58.569408 10311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:58.579025 10316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:58.579025 10319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:58.579059 10317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:58.579350 10311 server_base.cc:1034] running on GCE node
I20260811 22:20:58.580206 10311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:58.581594 10311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:58.582864 10311 hybrid_clock.cc:648] HybridClock initialized: now 1786486858582748 us; error 41 us; skew 500 ppm
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:58.586117 10311 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:20:58.587211 10311 webserver.cc:460] Webserver started at http://127.17.198.62:34265/ using document root <none> and password file <none>
I20260811 22:20:58.587841 10311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:58.587910 10311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:58.588200 10311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:58.590473 10311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "637f4b8537664f97ab80d127142563a1"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "73cb96142b922e1db98f51b5de58ba23"
server_key_iv: "81069d650cc7b2059bd672e808060415"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.591099 10311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "637f4b8537664f97ab80d127142563a1"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "73cb96142b922e1db98f51b5de58ba23"
server_key_iv: "81069d650cc7b2059bd672e808060415"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.595496 10311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:58.598428 10330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:58.599880 10311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:20:58.600059 10311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "637f4b8537664f97ab80d127142563a1"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "73cb96142b922e1db98f51b5de58ba23"
server_key_iv: "81069d650cc7b2059bd672e808060415"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.600205 10311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:58.628911 10311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:58.632519 10311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:58.632758 10311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:58.640637 10311 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:40783
I20260811 22:20:58.641010 10387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:40783 every 8 connection(s)
I20260811 22:20:58.641765 10311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:58.642553 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10311
I20260811 22:20:58.642711 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:20:58.642997 18200 external_mini_cluster.cc:1428] Setting key 59e1bc3e01b8043793a57b9ff4729009
I20260811 22:20:58.645078 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:58.649380 10388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.198.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.61@KRBTEST.COM" created.
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:58.667698 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.652032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40783 (local address 127.0.0.1:41148)
0811 22:20:58.652678 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.652723 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:58.653739 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:58.654501 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:58.654505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:58.654873 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:58.655377 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.655392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.657167 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.657170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.658020 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.658030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.658963 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.658967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.659074 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.659744 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:58.659764 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:58.662071 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:58.664357 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.664368 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.664380 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.664732 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.665229 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.665232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.665234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.665395 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.666110 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:58.666116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:58.666254 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 22:20:58.666416 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:20:58.666833 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":112,"thread_start_us":142,"threads_started":1}
I20260811 22:20:58.667698 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.652032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.17.198.62:40783)
0811 22:20:58.652560 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:58.652578 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:58.652867 (+ 289us) server_negotiation.cc:407] Connection header received
0811 22:20:58.654050 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:58.654055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:58.654244 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:58.654451 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:58.655552 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.657027 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.658159 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.658785 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.658926 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.662312 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:58.662318 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:58.662331 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:58.662355 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:58.664220 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.664984 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.664989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.664991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.665115 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.665528 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.665530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.665531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.665986 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:58.666105 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:58.666511 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 22:20:58.666788 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":366,"spinlock_wait_cycles":24960,"thread_start_us":183,"threads_started":1}
I20260811 22:20:58.670318 10388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:20:58.671356 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.671039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48764)
0811 22:20:58.671193 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.671274 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 22:20:58.671677 10334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
I20260811 22:20:58.673038 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.672658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48770)
0811 22:20:58.672783 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.672971 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:20:58.673295 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:34359: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
I20260811 22:20:58.678193 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:34359
--webserver_interface=127.17.198.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:20:58.722652 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } attempt: 1
I20260811 22:20:58.723690 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.723312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48776)
0811 22:20:58.723477 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.723561 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:20:58.724516 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.724267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48786)
0811 22:20:58.724386 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.724451 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:20:58.724669 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:34359: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
I20260811 22:20:58.792977 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } attempt: 2
I20260811 22:20:58.793905 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.793569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48798)
0811 22:20:58.793731 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.793822 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:20:58.794672 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.794451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:48808)
0811 22:20:58.794566 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.794619 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:20:58.794823 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:34359: Network error: Client connection negotiation failed: client connection to 127.17.198.61:34359: connect: Connection refused (error 111)
W20260811 22:20:58.809156 10395 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:20:58.809499 10395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:58.809568 10395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:58.809679 10395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:58.814172 10395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:58.814261 10395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:58.814284 10395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:58.814306 10395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:58.814347 10395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:58.820135 10395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:34359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.198.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:58.821470 10395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:58.822640 10395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:58.832190 10405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:58.832190 10401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:58.832275 10402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:58.832574 10395 server_base.cc:1034] running on GCE node
I20260811 22:20:58.833341 10395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:58.834661 10395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:58.835880 10395 hybrid_clock.cc:648] HybridClock initialized: now 1786486858835851 us; error 50 us; skew 500 ppm
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:58.839568 10395 init.cc:377] Logged in from keytab as kudu/127.17.198.61@KRBTEST.COM (short username kudu)
I20260811 22:20:58.840620 10395 webserver.cc:460] Webserver started at http://127.17.198.61:39321/ using document root <none> and password file <none>
I20260811 22:20:58.841305 10395 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:58.841385 10395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:58.841581 10395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:58.844007 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data/instance:
uuid: "70bccebab18e42f68f277c8eca12693b"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "32438ed937da1ca124f9bf9c98440f9e"
server_key_iv: "fdda6216aecf29b23586d5e232a379aa"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.844635 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal/instance:
uuid: "70bccebab18e42f68f277c8eca12693b"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "32438ed937da1ca124f9bf9c98440f9e"
server_key_iv: "fdda6216aecf29b23586d5e232a379aa"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.849164 10395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:20:58.851768 10413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:58.853554 10395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:20:58.853709 10395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "70bccebab18e42f68f277c8eca12693b"
format_stamp: "Formatted at 2026-08-11 22:20:58 on dist-test-slave-77v9"
server_key: "32438ed937da1ca124f9bf9c98440f9e"
server_key_iv: "fdda6216aecf29b23586d5e232a379aa"
server_key_version: "encryptionkey@0"
I20260811 22:20:58.853840 10395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:58.871650 10395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:58.875331 10395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:58.875567 10395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:58.883481 10395 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.61:34359
I20260811 22:20:58.883826 10472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.61:34359 every 8 connection(s)
I20260811 22:20:58.884562 10395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
I20260811 22:20:58.886022 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } attempt: 3
I20260811 22:20:58.886997 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10395
I20260811 22:20:58.887156 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-1/wal/instance
I20260811 22:20:58.887501 18200 external_mini_cluster.cc:1428] Setting key 1869a4f31df0368b0ed395b6b26e25b4
I20260811 22:20:58.889057 10474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:58.893561 10474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.198.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.60@KRBTEST.COM" created.
I20260811 22:20:58.902626 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.886647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:53380)
0811 22:20:58.886899 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.886921 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:58.887041 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:58.890010 (+ 2969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:58.890014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:58.890251 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:58.890422 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.890432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.892621 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.892624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.893497 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.893511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.894569 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.894573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.894697 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.895339 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:58.895353 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:58.897208 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:58.900122 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.900126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.900127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.900392 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.900916 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.900919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.900921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.901014 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.901854 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:58.901878 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:58.902017 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 22:20:58.902133 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:20:58.902327 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:58.904068 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.887521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.17.198.61:34359)
0811 22:20:58.888041 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:58.888065 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:58.888110 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:20:58.889210 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:58.889234 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:58.889626 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:58.890079 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:58.890593 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.892432 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.893657 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.894412 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.894616 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.897442 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:58.897449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:58.897461 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:58.897496 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:58.899935 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.900545 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.900557 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.900563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.900701 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.901128 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.901132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.901134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.901669 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:58.901838 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:58.902802 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 22:20:58.903137 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":154,"threads_started":1}
I20260811 22:20:58.907087 10388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:20:58.908061 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.895870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40783 (local address 127.0.0.1:57972)
0811 22:20:58.896352 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.896407 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:58.896599 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:58.897090 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:58.897094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:58.897569 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:58.897785 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.897797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.899019 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.899024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.900158 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.900171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.900708 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.900712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.900879 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.901494 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:58.901507 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:58.903239 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:58.905629 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.905642 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.905653 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.906044 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.906478 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.906498 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.906500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.906601 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.907109 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:58.907118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:58.907284 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 22:20:58.907468 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 22:20:58.907719 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":197,"threads_started":1}
I20260811 22:20:58.908080 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.907643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50498)
0811 22:20:58.907878 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.907967 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 22:20:58.909001 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.895926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.17.198.62:40783)
0811 22:20:58.896065 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:58.896070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:58.896501 (+ 431us) server_negotiation.cc:407] Connection header received
0811 22:20:58.896720 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:58.896724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:58.896808 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:58.896891 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:58.897946 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.898840 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.900284 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.900573 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.900701 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.903422 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:58.903429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:58.903435 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:58.903471 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:58.905499 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.906225 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.906229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.906231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.906340 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.906708 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.906712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.906714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.906980 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:58.907110 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:58.908689 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0811 22:20:58.908871 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:20:58.909540 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.908979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50510)
0811 22:20:58.909382 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.909448 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325}
W20260811 22:20:58.909734 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:58.910070 10474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:20:58 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:20:58.921180 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.910721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.17.198.61:34359)
0811 22:20:58.910815 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:58.910819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:58.910939 (+ 120us) server_negotiation.cc:407] Connection header received
0811 22:20:58.911098 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:58.911102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:58.911144 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:58.911247 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:58.911723 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.912307 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.913252 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.913528 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.913651 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.917392 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:58.917395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:58.917398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:58.917425 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:58.919034 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.919585 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.919588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.919589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.919674 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:58.920117 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:58.920120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:58.920121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:58.920348 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:58.920469 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:58.920857 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 22:20:58.921036 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 22:20:58.921187 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.910665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:53382)
0811 22:20:58.910861 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.910873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:58.910989 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:58.911256 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:58.911260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:58.911494 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:58.911598 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.911604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.912427 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.912434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.913121 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:58.913132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:58.913655 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:58.913659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:58.913762 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:58.914336 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:58.914355 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:58.917162 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:58.919196 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.919204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.919209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.919442 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.919859 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:58.919862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:58.919864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:58.920001 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:58.920505 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:58.920512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:58.920620 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 22:20:58.920711 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:20:58.920974 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 22:20:58.922559 10474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:20:58.923463 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.923147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50524)
0811 22:20:58.923298 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.923394 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:20:58.923725 10417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
I20260811 22:20:58.925073 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.924684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50534)
0811 22:20:58.924920 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.924986 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:20:58.925261 10474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:58.925851 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:42197
--webserver_interface=127.17.198.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:20:58.940871 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 1
I20260811 22:20:58.941869 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.941499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50538)
0811 22:20:58.941678 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.941754 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:20:58.942739 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.942475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50550)
0811 22:20:58.942623 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.942678 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 22:20:58.942926 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:58.974622 10474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 1
I20260811 22:20:58.975613 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.975264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50562)
0811 22:20:58.975447 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.975522 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:20:58.976504 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:58.976278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50564)
0811 22:20:58.976388 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:58.976442 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:20:58.976763 10474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:59.018162 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 2
I20260811 22:20:59.019244 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.018859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50570)
0811 22:20:59.019061 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.019142 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:20:59.020220 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.019946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50574)
0811 22:20:59.020087 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.020147 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:20:59.020406 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:59.045023 10474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 2
I20260811 22:20:59.046037 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.045672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50582)
0811 22:20:59.045862 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.045941 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:20:59.046945 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.046715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50588)
0811 22:20:59.046832 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.046884 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:20:59.047187 10474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
W20260811 22:20:59.047736 10481 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:20:59.048051 10481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:59.048120 10481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:59.048180 10481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:59.052618 10481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:59.052704 10481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:59.052735 10481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:59.052775 10481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:59.052839 10481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:59.058496 10481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:40783,127.17.198.61:34359,127.17.198.60:42197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:42197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.198.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:20:59.059686 10481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:59.060850 10481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:59.070034 10486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:59.070086 10489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:59.070199 10481 server_base.cc:1034] running on GCE node
W20260811 22:20:59.070075 10487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:59.071084 10481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:59.072368 10481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:59.073580 10481 hybrid_clock.cc:648] HybridClock initialized: now 1786486859073554 us; error 35 us; skew 500 ppm
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:59.076658 10481 init.cc:377] Logged in from keytab as kudu/127.17.198.60@KRBTEST.COM (short username kudu)
I20260811 22:20:59.077715 10481 webserver.cc:460] Webserver started at http://127.17.198.60:34789/ using document root <none> and password file <none>
I20260811 22:20:59.078331 10481 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:59.078439 10481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:59.078680 10481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:20:59.080886 10481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data/instance:
uuid: "8e2de92de201405e853f1136cc44b9dd"
format_stamp: "Formatted at 2026-08-11 22:20:59 on dist-test-slave-77v9"
server_key: "cddeea5ddd5c8065834d3ec9cbfe8c52"
server_key_iv: "88e822987e5beda89819ac396d56e1f7"
server_key_version: "encryptionkey@0"
I20260811 22:20:59.081586 10481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal/instance:
uuid: "8e2de92de201405e853f1136cc44b9dd"
format_stamp: "Formatted at 2026-08-11 22:20:59 on dist-test-slave-77v9"
server_key: "cddeea5ddd5c8065834d3ec9cbfe8c52"
server_key_iv: "88e822987e5beda89819ac396d56e1f7"
server_key_version: "encryptionkey@0"
I20260811 22:20:59.085737 10481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:20:59.088462 10498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:59.089751 10481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 22:20:59.089920 10481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "8e2de92de201405e853f1136cc44b9dd"
format_stamp: "Formatted at 2026-08-11 22:20:59 on dist-test-slave-77v9"
server_key: "cddeea5ddd5c8065834d3ec9cbfe8c52"
server_key_iv: "88e822987e5beda89819ac396d56e1f7"
server_key_version: "encryptionkey@0"
I20260811 22:20:59.090090 10481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:59.119594 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 3
I20260811 22:20:59.120564 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.120238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50600)
0811 22:20:59.120403 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.120479 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:20:59.121351 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.121123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50610)
0811 22:20:59.121242 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.121295 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:20:59.121510 10388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:59.129719 10481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:59.133112 10481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:59.133340 10481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:59.138420 10474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 3
I20260811 22:20:59.139616 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.139130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50624)
0811 22:20:59.139446 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.139521 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260811 22:20:59.140592 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.140205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50636)
0811 22:20:59.140429 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.140503 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 22:20:59.140787 10474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:42197: Network error: Client connection negotiation failed: client connection to 127.17.198.60:42197: connect: Connection refused (error 111)
I20260811 22:20:59.142112 10481 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.60:42197
I20260811 22:20:59.142423 10561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.60:42197 every 8 connection(s)
I20260811 22:20:59.143282 10481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
I20260811 22:20:59.144438 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10481
I20260811 22:20:59.144562 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486470476166-18200-0/minicluster-data/master-2/wal/instance
I20260811 22:20:59.144896 18200 external_mini_cluster.cc:1428] Setting key e7f4c077f776aa4fa96714e3e1d4a678
I20260811 22:20:59.146873 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:59.150779 10563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:20:59.161985 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.146588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.17.198.60:42197)
0811 22:20:59.147096 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.147114 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.147151 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:20:59.147880 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.147897 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.148247 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.148600 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.149609 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.150664 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.151669 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.152257 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.152443 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.156073 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.156078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.156088 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.156120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.158553 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.159049 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.159058 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.159067 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.159222 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.159601 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.159605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.159607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.160154 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.160304 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.160735 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.161126 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":140,"threads_started":1}
I20260811 22:20:59.164355 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.152660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.17.198.62:40783)
0811 22:20:59.152901 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.152906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.153331 (+ 425us) server_negotiation.cc:407] Connection header received
0811 22:20:59.153540 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.153542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.153591 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.153670 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.154640 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.155489 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.156632 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.157100 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.157222 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.160037 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.160040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.160042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.160069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.162002 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.162621 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.162624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.162625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.162733 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.163193 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.163198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.163199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.163458 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.163574 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.164024 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.164180 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:20:59.164364 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.152558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:40783 (local address 127.0.0.1:57978)
0811 22:20:59.153147 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.153228 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:59.153417 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:59.153757 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:59.153764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:59.154261 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:59.154488 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.154499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.155648 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.155654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.156483 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.156496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.157251 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.157261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.157415 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.157955 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:59.157967 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:59.159857 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:59.162150 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.162161 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.162168 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.162488 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.162899 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.162902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.162904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.163011 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.163583 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:59.163588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:59.163703 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:20:59.163914 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:20:59.164134 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":287,"threads_started":1}
I20260811 22:20:59.166061 10563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:20:59.175040 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.165025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.17.198.62:40783)
0811 22:20:59.165137 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.165141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.165173 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:20:59.165366 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.165372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.165424 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.165517 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.166335 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.167051 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.168096 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.168330 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.168465 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.171533 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.171536 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.171538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.171561 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.173322 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.173768 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.173772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.173773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.173864 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.174148 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.174153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.174154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.174368 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.174613 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.174735 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.174871 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:20:59.175344 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.166682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:34359 (local address 127.0.0.1:53398)
0811 22:20:59.166796 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.166810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:59.166900 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:59.167217 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:59.167219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:59.167366 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:59.167471 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.167476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.168363 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.168368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.168983 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.168992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.169459 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.169463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.169564 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.170036 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:59.170049 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:59.171708 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:59.173647 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.173652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.173655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.173877 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.174206 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.174209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.174211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.174308 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.174737 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:59.174741 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:59.174853 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:20:59.174981 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:20:59.175148 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 22:20:59.175765 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.166818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.17.198.61:34359)
0811 22:20:59.166994 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.166999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.167010 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:20:59.167054 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.167056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.167118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.167203 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.167589 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.168249 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.169105 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.169337 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.169443 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.171882 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.171886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.171888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.171915 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.173513 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.173990 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.173996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.173998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.174090 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.174412 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.174416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.174428 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.174623 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.174744 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.175186 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.175599 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:20:59.176604 10563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:20:59.185572 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.177221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50656)
0811 22:20:59.177331 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.177348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:59.177440 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:59.177779 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:59.177782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:59.177969 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:59.178039 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.178044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.178833 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.178840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.179399 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.179407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.179891 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.179895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.179991 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.180459 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:59.180471 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:59.182225 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:59.183959 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.183963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.183968 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.184171 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.184459 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.184462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.184464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.184544 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.184984 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:59.184986 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:59.185079 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 22:20:59.185184 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:20:59.185336 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 22:20:59.185572 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.177508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.17.198.60:42197)
0811 22:20:59.177611 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.177614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.177623 (+ 9us) server_negotiation.cc:407] Connection header received
0811 22:20:59.177647 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.177649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.177678 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.177766 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.178160 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.178719 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.179522 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.179767 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.179880 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.182378 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.182382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.182384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.182405 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.183785 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.184273 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.184278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.184280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.184355 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.184633 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.184638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.184640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.184870 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.184994 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.185239 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.185395 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":33}
I20260811 22:20:59.185637 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.176406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.17.198.61:34359)
0811 22:20:59.176526 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.176530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.176542 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:20:59.176665 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.176669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.176715 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.176788 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.177591 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.178092 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.178924 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.179156 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.179270 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.182111 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.182115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.182117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.182139 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.183729 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.184174 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.184179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.184181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.184262 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.184562 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.184566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.184568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.184759 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.185088 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.185230 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.185362 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:20:59.189677 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:20:59.189744 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: Bootstrap starting.
I20260811 22:20:59.191330 10361 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "637f4b8537664f97ab80d127142563a1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57986
I20260811 22:20:59.192493 10563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:59.193531 10563 log.cc:824] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:59.195922 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: No bootstrap required, opened a new log
I20260811 22:20:59.199206 10563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.199429 10563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2de92de201405e853f1136cc44b9dd, State: Initialized, Role: FOLLOWER
I20260811 22:20:59.199790 10563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.200316 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.200466 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.201300 10563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:20:59.206549 10582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:59.206677 10582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:59.205639 10481 master_runner.cc:186] Error getting master registration for 127.17.198.62:40783: OK, instance_id { permanent_uuid: "637f4b8537664f97ab80d127142563a1" instance_seqno: 1786486858638347 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:20:59.208505 10481 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:20:59.284008 10474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 4
I20260811 22:20:59.285682 10388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } attempt: 4
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
Aug 11 22:20:59 dist-test-slave-77v9 krb5kdc[10294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486858, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:20:59.294418 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.284683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50668)
0811 22:20:59.284864 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.284883 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:59.285001 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:59.285321 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:59.285324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:59.285531 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:59.285647 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.285657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.286755 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.286758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.287441 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.287452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.288104 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.288107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.288190 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.288734 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:59.288746 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:59.290504 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:59.292584 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.292588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.292590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.292888 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.293226 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.293230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.293232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.293333 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.293878 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:59.293882 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:59.293988 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:20:59.294080 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:20:59.294233 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:20:59.295112 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.284776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.17.198.60:42197)
0811 22:20:59.284935 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.284948 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.284966 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:20:59.285138 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.285142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.285198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.285291 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.285810 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.286615 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.287575 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.287894 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.287985 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.290658 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.290663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.290668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.290700 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.292458 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.292989 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.292992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.292994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.293103 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.293437 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.293441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.293443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.293661 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.293839 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.294689 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.294952 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"spinlock_wait_cycles":10112}
I20260811 22:20:59.297719 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.286195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:42197 (local address 127.0.0.1:50674)
0811 22:20:59.286410 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:59.286428 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:59.286531 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:59.286996 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:59.286999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:59.287207 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:59.287319 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.287325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.288309 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.288312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.289526 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:59.289539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.290174 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.290178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:59.290270 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.290866 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:59.290881 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:59.292586 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:59.296038 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.296041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.296043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.296328 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.296614 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:59.296618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:59.296620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:59.296717 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:59.297169 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:59.297172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:59.297281 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 22:20:59.297393 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:20:59.297554 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 22:20:59.297737 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.286301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.17.198.60:42197)
0811 22:20:59.286672 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.286683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.286697 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:20:59.286734 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.286738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.286780 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.286973 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.287444 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.288216 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.289664 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.290039 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.290153 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.293048 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.293051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.293054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.293098 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.295678 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.296436 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.296439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.296440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.296511 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.296843 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.296848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.296851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.297052 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.297170 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.297447 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.297564 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"spinlock_wait_cycles":3712,"thread_start_us":143,"threads_started":1}
I20260811 22:20:59.298961 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: Bootstrap starting.
I20260811 22:20:59.300279 10388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: Bootstrap starting.
I20260811 22:20:59.301893 10474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:59.302745 10388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:59.302910 10474 log.cc:824] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:59.303568 10388 log.cc:824] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:59.305632 10388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: No bootstrap required, opened a new log
I20260811 22:20:59.305635 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: No bootstrap required, opened a new log
I20260811 22:20:59.308842 10388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.309106 10388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637f4b8537664f97ab80d127142563a1, State: Initialized, Role: FOLLOWER
I20260811 22:20:59.309456 10388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.309639 10474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.309868 10474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70bccebab18e42f68f277c8eca12693b, State: Initialized, Role: FOLLOWER
I20260811 22:20:59.310154 10474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.310163 10590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.310524 10590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.310987 10388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:59.311019 10361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:20:59.311112 10361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:59.311333 10474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:59.312592 10361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.313686 10361 leader_election.cc:288] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [CANDIDATE]: Term 1 election: Requested vote from peers 70bccebab18e42f68f277c8eca12693b (127.17.198.61:34359), 8e2de92de201405e853f1136cc44b9dd (127.17.198.60:42197)
I20260811 22:20:59.314495 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.314657 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.315248 10530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "637f4b8537664f97ab80d127142563a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e2de92de201405e853f1136cc44b9dd"
I20260811 22:20:59.315546 10530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:59.317143 10530 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 637f4b8537664f97ab80d127142563a1 in term 1.
W20260811 22:20:59.317410 10614 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:20:59.317483 10614 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:59.316747 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "637f4b8537664f97ab80d127142563a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "70bccebab18e42f68f277c8eca12693b"
I20260811 22:20:59.319161 10445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:59.321651 10334 leader_election.cc:302] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 637f4b8537664f97ab80d127142563a1, 8e2de92de201405e853f1136cc44b9dd; no voters:
I20260811 22:20:59.328197 10590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:59.328605 10590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [term 1 LEADER]: Becoming Leader. State: Replica: 637f4b8537664f97ab80d127142563a1, State: Running, Role: LEADER
W20260811 22:20:59.320782 10615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 22:20:59.320922 10445 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 637f4b8537664f97ab80d127142563a1 in term 1.
W20260811 22:20:59.328966 10615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:20:59.334285 10395 master_runner.cc:263] No leader master found from master 70bccebab18e42f68f277c8eca12693b
I20260811 22:20:59.334378 10395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:20:59.335481 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.321171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.17.198.61:34359)
0811 22:20:59.321291 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.321297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.321314 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:20:59.321357 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.321359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.321410 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.321480 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.322539 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.323271 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.324398 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.324719 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.324797 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.326825 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.326830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.326833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.326859 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.332700 (+ 5841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.334022 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.334026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.334028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.334126 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.334457 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.334463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.334464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.334693 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.334796 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.335206 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.335335 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:20:59.339188 10311 master_runner.cc:263] No leader master found from master 637f4b8537664f97ab80d127142563a1
I20260811 22:20:59.339262 10311 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:20:59.339612 10590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } }
I20260811 22:20:59.341015 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.321468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.17.198.62:40783)
0811 22:20:59.322546 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.322550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.322563 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:20:59.322599 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.322602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.322641 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.322713 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.326670 (+ 3957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.327365 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.328139 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.328438 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.328550 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.330721 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.330724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.330727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.330755 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.333505 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.337689 (+ 4184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.337692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.337694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.337789 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.339928 (+ 2139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.339931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.339933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.340161 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.340317 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.340691 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.340862 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987}
I20260811 22:20:59.343372 10625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 637f4b8537664f97ab80d127142563a1. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.343541 10625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:59.344533 10627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:59.345781 10627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:59.347942 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:59.319836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.17.198.60:42197)
0811 22:20:59.321014 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.321018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.336171 (+ 15153us) server_negotiation.cc:407] Connection header received
0811 22:20:59.336365 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.336369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.336424 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.336476 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.338137 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.338853 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.341669 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:59.342011 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:59.342090 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:59.344038 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:59.344043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:59.344045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:59.344072 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:59.345893 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.346399 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.346403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.346405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.346524 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:59.346856 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:59.346859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:59.346861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:59.347092 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:59.347227 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:59.347583 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 22:20:59.347730 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101}
I20260811 22:20:59.356716 10445 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [term 1 FOLLOWER]: Refusing update from remote peer 637f4b8537664f97ab80d127142563a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:59.357642 10590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:59.357826 10530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [term 1 FOLLOWER]: Refusing update from remote peer 637f4b8537664f97ab80d127142563a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:59.359308 10625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:59.362396 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 637f4b8537664f97ab80d127142563a1. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.362550 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.363231 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 637f4b8537664f97ab80d127142563a1. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.363348 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.368911 10590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.369074 10590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:59.370098 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.370229 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.370682 10628 mvcc.cc:205] Tried to move back new op lower bound from 7317450175917092864 to 7317450175859798016. Current Snapshot: MvccSnapshot[applied={T|T < 7317450175917092864}]
I20260811 22:20:59.371258 10630 mvcc.cc:205] Tried to move back new op lower bound from 7317450175917092864 to 7317450175859798016. Current Snapshot: MvccSnapshot[applied={T|T < 7317450175917092864}]
I20260811 22:20:59.374994 10627 catalog_manager.cc:1347] Generated new cluster ID: 17ed89e9577b40ae99560e6598f008c7
I20260811 22:20:59.375083 10627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 22:20:59.377432 10419 tablet.cc:2369] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:59.380568 10625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.380704 10625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:59.382112 10591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "637f4b8537664f97ab80d127142563a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637f4b8537664f97ab80d127142563a1" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 40783 } } peers { permanent_uuid: "70bccebab18e42f68f277c8eca12693b" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 34359 } } peers { permanent_uuid: "8e2de92de201405e853f1136cc44b9dd" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 42197 } } }
I20260811 22:20:59.382243 10591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:20:59.383147 10629 mvcc.cc:205] Tried to move back new op lower bound from 7317450175917092864 to 7317450175859798016. Current Snapshot: MvccSnapshot[applied={T|T < 7317450175917092864}]
I20260811 22:20:59.392019 10627 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:59.394094 10627 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:59.408401 10627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 637f4b8537664f97ab80d127142563a1: Generated new TSK 0
I20260811 22:20:59.409379 10627 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:20:59.455149 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.452887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.17.198.61:34359)
0811 22:20:59.453249 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.453254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.453953 (+ 699us) server_negotiation.cc:407] Connection header received
0811 22:20:59.454052 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.454057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.454112 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.454198 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.454960 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53418: BlockingRecv error: recv got EOF from 127.0.0.1:53418 (error 108)
Metrics: {"server-negotiator.queue_time_us":242}
W20260811 22:20:59.455400 10586 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.453327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.17.198.60:42197)
0811 22:20:59.453505 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.453509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.454254 (+ 745us) server_negotiation.cc:407] Connection header received
0811 22:20:59.454477 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.454482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.454541 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.454599 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.455257 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50690: BlockingRecv error: recv got EOF from 127.0.0.1:50690 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 22:20:59.456625 10392 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.453358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58008 (local address 127.17.198.62:40783)
0811 22:20:59.453619 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.453624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.453910 (+ 286us) server_negotiation.cc:407] Connection header received
0811 22:20:59.455865 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.455873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.455925 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.456026 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.456485 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58008: BlockingRecv error: recv got EOF from 127.0.0.1:58008 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
W20260811 22:20:59.460199 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.458898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.17.198.61:34359)
0811 22:20:59.459026 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.459031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.459314 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:20:59.459510 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.459513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.459589 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.459680 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.460095 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53420: BlockingRecv error: recv got EOF from 127.0.0.1:53420 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:20:59.466149 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.463770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.17.198.61:34359)
0811 22:20:59.463903 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.463908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.465180 (+ 1272us) server_negotiation.cc:407] Connection header received
0811 22:20:59.465372 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.465375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.465434 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.465534 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.466024 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53422: BlockingRecv error: recv got EOF from 127.0.0.1:53422 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 22:20:59.471875 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.470537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.17.198.61:34359)
0811 22:20:59.470670 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.470675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.470942 (+ 267us) server_negotiation.cc:407] Connection header received
0811 22:20:59.471117 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.471121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.471177 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.471269 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.471745 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53436: BlockingRecv error: recv got EOF from 127.0.0.1:53436 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:20:59.480149 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.478699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.17.198.61:34359)
0811 22:20:59.478885 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.478891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.479158 (+ 267us) server_negotiation.cc:407] Connection header received
0811 22:20:59.479338 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.479349 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.479400 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.479502 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.480012 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53446: BlockingRecv error: recv got EOF from 127.0.0.1:53446 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 22:20:59.492113 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.490599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.17.198.61:34359)
0811 22:20:59.490760 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.490765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.491077 (+ 312us) server_negotiation.cc:407] Connection header received
0811 22:20:59.491273 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.491277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.491342 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.491449 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.491974 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53448: BlockingRecv error: recv got EOF from 127.0.0.1:53448 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:20:59.512013 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.510365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.17.198.61:34359)
0811 22:20:59.510515 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.510520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.510877 (+ 357us) server_negotiation.cc:407] Connection header received
0811 22:20:59.511057 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.511062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.511118 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.511258 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.511875 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53464: BlockingRecv error: recv got EOF from 127.0.0.1:53464 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 22:20:59.547520 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.546144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.17.198.61:34359)
0811 22:20:59.546283 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.546288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.546585 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:20:59.546766 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.546770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.546821 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.546913 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.547397 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53478: BlockingRecv error: recv got EOF from 127.0.0.1:53478 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:20:59.615312 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.613869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.17.198.61:34359)
0811 22:20:59.614096 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.614101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.614245 (+ 144us) server_negotiation.cc:407] Connection header received
0811 22:20:59.614366 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.614368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.614419 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.614496 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.615154 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53494: BlockingRecv error: recv got EOF from 127.0.0.1:53494 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 22:20:59.747419 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:20:59.746023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.17.198.61:34359)
0811 22:20:59.746232 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:59.746237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:59.746256 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:20:59.746458 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:59.746462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:59.746534 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:59.746627 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:59.747253 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53498: BlockingRecv error: recv got EOF from 127.0.0.1:53498 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 22:21:00.007213 10476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:00.005967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.17.198.61:34359)
0811 22:21:00.006157 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.006161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.006186 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:21:00.006345 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.006349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.006403 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.006514 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:00.007087 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53500: BlockingRecv error: recv got EOF from 127.0.0.1:53500 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:21:00.208148 10582 catalog_manager.cc:1259] Loaded cluster ID: 17ed89e9577b40ae99560e6598f008c7
I20260811 22:21:00.208247 10582 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: loading cluster ID for follower catalog manager: success
I20260811 22:21:00.210142 10582 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: acquiring CA information for follower catalog manager: success
I20260811 22:21:00.211714 10582 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e2de92de201405e853f1136cc44b9dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:00.330493 10615 catalog_manager.cc:1259] Loaded cluster ID: 17ed89e9577b40ae99560e6598f008c7
I20260811 22:21:00.330651 10615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: loading cluster ID for follower catalog manager: success
I20260811 22:21:00.332491 10615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: acquiring CA information for follower catalog manager: success
I20260811 22:21:00.333823 10615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70bccebab18e42f68f277c8eca12693b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:00.526001 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:00.521510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.17.198.61:34359)
0811 22:21:00.521825 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.521830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.521898 (+ 68us) server_negotiation.cc:407] Connection header received
0811 22:21:00.522119 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.522122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.522276 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.522405 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:00.522649 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.523299 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.524225 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.524488 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.524581 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:00.524993 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:00.525472 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:00.525576 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:00.525636 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:21:00.525691 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260811 22:21:00.533656 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:00.529613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.17.198.60:42197)
0811 22:21:00.529885 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.529890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.529905 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:00.530145 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.530147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.530275 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.530402 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:00.530648 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.531177 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.532024 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.532259 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.532369 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:00.532705 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:00.533226 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:00.533379 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:00.533417 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 22:21:00.533488 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260811 22:21:00.541225 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:00.537429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.17.198.60:42197)
0811 22:21:00.537577 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.537582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.537647 (+ 65us) server_negotiation.cc:407] Connection header received
0811 22:21:00.537736 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.537738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.537792 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.537868 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:00.538113 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.538846 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.539933 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.540224 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.540325 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:00.540663 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:00.540756 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:00.540894 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:00.541018 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:21:00.541087 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:21:00.541939 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:00.537011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.17.198.61:34359)
0811 22:21:00.537185 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.537190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.537447 (+ 257us) server_negotiation.cc:407] Connection header received
0811 22:21:00.538255 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.538259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.538312 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.538393 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:00.538655 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.539416 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.540569 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.540981 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.541074 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:00.541453 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:00.541533 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:00.541579 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:00.541739 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:21:00.541812 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:00.542357 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:00.536864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.17.198.62:40783)
0811 22:21:00.537623 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:00.537627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:00.537643 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:21:00.537747 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:00.537749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:00.537915 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:00.538044 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:00.538344 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.539058 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.540198 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:00.540529 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:00.540670 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:00.541111 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:00.541489 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:00.541624 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:00.542148 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 22:21:00.542219 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":96,"threads_started":1}
I20260811 22:21:00.546103 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10311
I20260811 22:21:00.556721 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10395
I20260811 22:21:00.565554 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10481
2026-08-11T22:21:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3430 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:00 dist-test-slave-77v9 krb5kdc[10736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:00 dist-test-slave-77v9 krb5kdc[10736](info): set up 2 sockets
Aug 11 22:21:00 dist-test-slave-77v9 krb5kdc[10736](info): commencing operation
krb5kdc: starting...
W20260811 22:21:01.747804 18200 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/test-user.keytab.
Aug 11 22:21:01 dist-test-slave-77v9 krb5kdc[10736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.198.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
Entry for principal kudu/127.17.198.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab.
I20260811 22:21:01.896932 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42835
--webserver_interface=127.17.198.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:02.018571 10751 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:21:02.019057 10751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:02.019150 10751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:02.019235 10751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:02.023543 10751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:02.023672 10751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:02.023707 10751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:02.023746 10751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:02.023780 10751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:02.029521 10751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.62.keytab
--principal=kudu/127.17.198.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.62:42835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.198.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:02.030792 10751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:02.031934 10751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:02.041260 10759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:02.041275 10757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:02.041334 10751 server_base.cc:1034] running on GCE node
W20260811 22:21:02.041260 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:02.042244 10751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:02.043524 10751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:02.044708 10751 hybrid_clock.cc:648] HybridClock initialized: now 1786486862044682 us; error 54 us; skew 500 ppm
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:02.047830 10751 init.cc:377] Logged in from keytab as kudu/127.17.198.62@KRBTEST.COM (short username kudu)
I20260811 22:21:02.048869 10751 webserver.cc:460] Webserver started at http://127.17.198.62:43301/ using document root <none> and password file <none>
I20260811 22:21:02.049476 10751 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:02.049576 10751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:02.049823 10751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:02.052107 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data/instance:
uuid: "2efa4d885dd04b46a2dc07b07629fdc4"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "19d7b2a2bf7f43ee8d66d72d27c71ac0"
server_key_iv: "026eadb6ee2b4d9e9e938aecf61f37ef"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.052740 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance:
uuid: "2efa4d885dd04b46a2dc07b07629fdc4"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "19d7b2a2bf7f43ee8d66d72d27c71ac0"
server_key_iv: "026eadb6ee2b4d9e9e938aecf61f37ef"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.057101 10751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:02.059796 10771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:02.061235 10751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:02.061398 10751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal
uuid: "2efa4d885dd04b46a2dc07b07629fdc4"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "19d7b2a2bf7f43ee8d66d72d27c71ac0"
server_key_iv: "026eadb6ee2b4d9e9e938aecf61f37ef"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.061537 10751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:02.082742 10751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:02.086369 10751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:02.086618 10751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:02.094260 10751 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.62:42835
I20260811 22:21:02.094530 10831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.62:42835 every 8 connection(s)
I20260811 22:21:02.095414 10751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:02.096009 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10751
I20260811 22:21:02.096143 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-0/wal/instance
I20260811 22:21:02.096411 18200 external_mini_cluster.cc:1428] Setting key 33fd9888955569c4a74cfd070ded30ea
I20260811 22:21:02.098759 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:02.103904 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.198.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.61@KRBTEST.COM" created.
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:21:02.121081 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.106568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.17.198.62:42835)
0811 22:21:02.107144 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.107162 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.107442 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:21:02.108617 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.108621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.108924 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.109088 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.110221 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.111621 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.112650 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.113131 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.113351 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.115827 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.115833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.115844 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.115865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.117609 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.118136 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.118142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.118144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.118260 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.118590 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.118595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.118597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.119025 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.119173 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.119563 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.119911 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":403,"spinlock_wait_cycles":18816,"thread_start_us":201,"threads_started":1}
I20260811 22:21:02.121081 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.106568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42835 (local address 127.0.0.1:43002)
0811 22:21:02.107262 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.107317 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.108275 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.109126 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.109131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.109652 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.110048 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.110065 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.111762 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.111765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.112507 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.112515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.113304 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.113309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.113421 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.114082 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.114101 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.115649 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.117746 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.117757 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.117768 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.118017 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.118370 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.118373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.118375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.118495 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.119154 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.119160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.119308 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.119486 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.119964 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":157,"thread_start_us":131,"threads_started":1}
I20260811 22:21:02.123651 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:02.124533 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.124240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53426)
0811 22:21:02.124378 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.124459 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:21:02.124747 10775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
I20260811 22:21:02.125839 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.125534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53434)
0811 22:21:02.125693 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.125758 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:21:02.125993 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:44035: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
Entry for principal kudu/127.17.198.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab.
I20260811 22:21:02.129572 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:44035
--webserver_interface=127.17.198.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:02.175334 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } attempt: 1
I20260811 22:21:02.176407 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.175992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53438)
0811 22:21:02.176197 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.176299 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:21:02.177384 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.177104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53446)
0811 22:21:02.177229 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.177296 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:21:02.177595 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:44035: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
I20260811 22:21:02.245831 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } attempt: 2
I20260811 22:21:02.246959 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.246554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53456)
0811 22:21:02.246773 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.246854 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 22:21:02.247843 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.247579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53458)
0811 22:21:02.247698 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.247766 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:21:02.248020 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.61:44035: Network error: Client connection negotiation failed: client connection to 127.17.198.61:44035: connect: Connection refused (error 111)
W20260811 22:21:02.251740 10842 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:21:02.252053 10842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:02.252137 10842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:02.252235 10842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:02.256486 10842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:02.256570 10842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:02.256593 10842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:02.256614 10842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:02.256670 10842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:02.262508 10842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.61.keytab
--principal=kudu/127.17.198.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.61:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.198.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:02.263743 10842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:02.264918 10842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:02.275229 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:02.275275 10849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:02.275159 10847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:02.275484 10842 server_base.cc:1034] running on GCE node
I20260811 22:21:02.276160 10842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:02.277516 10842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:02.278896 10842 hybrid_clock.cc:648] HybridClock initialized: now 1786486862278886 us; error 36 us; skew 500 ppm
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:02.282065 10842 init.cc:377] Logged in from keytab as kudu/127.17.198.61@KRBTEST.COM (short username kudu)
I20260811 22:21:02.283016 10842 webserver.cc:460] Webserver started at http://127.17.198.61:39569/ using document root <none> and password file <none>
I20260811 22:21:02.283565 10842 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:02.283631 10842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:02.283818 10842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:02.286100 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data/instance:
uuid: "14d9766497ac4454bf5e34f25c2aec0a"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "b9df8d6be1c937450dafb17d72ddb3a1"
server_key_iv: "972124af81cebe6aee3174d89a0b1844"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.286672 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal/instance:
uuid: "14d9766497ac4454bf5e34f25c2aec0a"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "b9df8d6be1c937450dafb17d72ddb3a1"
server_key_iv: "972124af81cebe6aee3174d89a0b1844"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.290904 10842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:21:02.293491 10859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:02.294834 10842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:02.294968 10842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal
uuid: "14d9766497ac4454bf5e34f25c2aec0a"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "b9df8d6be1c937450dafb17d72ddb3a1"
server_key_iv: "972124af81cebe6aee3174d89a0b1844"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.295085 10842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:02.313406 10842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:02.317154 10842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:02.317348 10842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:02.325430 10842 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.61:44035
I20260811 22:21:02.325753 10919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.61:44035 every 8 connection(s)
I20260811 22:21:02.326589 10842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/data/info.pb
I20260811 22:21:02.328445 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10842
I20260811 22:21:02.328555 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-1/wal/instance
I20260811 22:21:02.328934 18200 external_mini_cluster.cc:1428] Setting key 93f5a741cbe31d6f27859b5758f7998b
I20260811 22:21:02.329835 10920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:02.334053 10920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.198.60@KRBTEST.COM; defaulting to no policy
I20260811 22:21:02.339232 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.198.60@KRBTEST.COM" created.
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:21:02.352118 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.339727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53472)
0811 22:21:02.339863 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.339879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.339964 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.340871 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.340873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.341079 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.341299 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.341307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.342790 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.342793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.343532 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.343541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.344283 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.344287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.344399 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.345137 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.345155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.346869 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.349853 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.349860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.349865 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.350156 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.350563 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.350572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.350681 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.351576 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.351578 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.351669 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.351774 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.351927 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":27}
I20260811 22:21:02.352936 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.336575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.17.198.62:42835)
0811 22:21:02.336762 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.336767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.337884 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 22:21:02.339053 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.339056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.339118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.339207 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.341153 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.341999 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.343384 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.343812 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.343942 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.348006 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.348012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.348014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.348044 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.349996 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350598 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.350602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.350604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.350691 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.351030 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.351033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.351035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.351232 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.351353 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.352544 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.352676 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":70}
I20260811 22:21:02.353266 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.337118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42835 (local address 127.0.0.1:43018)
0811 22:21:02.337674 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.337732 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.338809 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.339462 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.339474 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.340200 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.340992 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.341015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.342129 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.342135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.343231 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.343244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.343985 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.343989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.344206 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.345048 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.345072 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.347813 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.350117 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.350139 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.350470 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.350796 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.350801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.350927 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.351364 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.351374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.351541 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.351749 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.352100 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":22,"thread_start_us":176,"threads_started":1}
I20260811 22:21:02.353266 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.339890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.17.198.61:44035)
0811 22:21:02.340366 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.340377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.340403 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:02.340448 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.340453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.340638 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.340850 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.341440 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.342653 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.343660 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.344138 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.344591 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.347043 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.347050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.347063 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.347088 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.349681 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350301 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.350308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.350309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.350446 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.350812 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.350819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.350823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.351456 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.351603 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.352790 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.352971 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":374,"thread_start_us":181,"threads_started":1}
I20260811 22:21:02.355052 10920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:02.355551 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:02.356232 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.355963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35896)
0811 22:21:02.356089 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.356151 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 22:21:02.357048 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.356726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35898)
0811 22:21:02.356920 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.356978 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260811 22:21:02.357200 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
Entry for principal kudu/127.17.198.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab.
I20260811 22:21:02.364686 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.355651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53474)
0811 22:21:02.355937 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.355951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.356042 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.356291 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.356294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.356517 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.356633 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.356641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.357647 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.357650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.358240 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.358249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.358718 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.358721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.358860 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.359366 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.359379 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.361046 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.362974 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.362978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.362980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.363188 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.363486 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.363488 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.363490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.363596 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.364088 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.364093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.364210 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.364352 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.364502 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":22}
I20260811 22:21:02.365404 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.355876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.17.198.61:44035)
0811 22:21:02.355976 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.355983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.355998 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:21:02.356127 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.356130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.356173 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.356256 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.356743 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.357509 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.358360 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.358607 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.358721 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.361188 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.361191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.361194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.361224 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.362848 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.363292 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.363295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.363297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.363374 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.363693 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.363696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.363698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.363899 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.364032 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.365005 (+ 973us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.365158 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":28}
I20260811 22:21:02.365597 18200 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski7rXue/build/debug/bin/kudu
/tmp/dist-test-taski7rXue/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:44455
--webserver_interface=127.17.198.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.198.20:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:02.368062 10920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:02.369102 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.368645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35900)
0811 22:21:02.368799 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.369033 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:21:02.369760 10863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
I20260811 22:21:02.370784 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.370563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35908)
0811 22:21:02.370677 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.370733 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:21:02.370941 10920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
I20260811 22:21:02.388379 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 1
I20260811 22:21:02.389427 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.389052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35920)
0811 22:21:02.389238 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.389326 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:21:02.390241 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.390006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35924)
0811 22:21:02.390119 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.390177 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:21:02.390424 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
I20260811 22:21:02.420238 10920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 1
I20260811 22:21:02.421267 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.420945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35936)
0811 22:21:02.421103 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.421182 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:21:02.422142 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.421902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35950)
0811 22:21:02.422023 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.422082 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:21:02.422395 10920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
I20260811 22:21:02.465588 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 2
I20260811 22:21:02.466533 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.466174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35964)
0811 22:21:02.466352 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.466424 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:21:02.467434 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.467187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35968)
0811 22:21:02.467321 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.467374 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:21:02.467624 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
W20260811 22:21:02.487701 10926 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:21:02.488034 10926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:02.488058 10926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:02.488101 10926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:21:02.490587 10920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 2
I20260811 22:21:02.491671 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.491292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35974)
0811 22:21:02.491507 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.491582 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 22:21:02.492597 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.492347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:35990)
0811 22:21:02.492472 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.492531 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:21:02.492736 10920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
W20260811 22:21:02.492959 10926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:02.493038 10926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:02.493086 10926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:02.493117 10926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:02.493144 10926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:02.498888 10926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.198.20:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.198.62:42835,127.17.198.61:44035,127.17.198.60:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/krb5kdc/kudu_127.17.198.60.keytab
--principal=kudu/127.17.198.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.198.60:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.198.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:21:02.500451 10926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:02.501600 10926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:02.510905 10937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:02.510881 10933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:02.510881 10934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:02.511119 10926 server_base.cc:1034] running on GCE node
I20260811 22:21:02.512065 10926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:02.513396 10926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:02.514593 10926 hybrid_clock.cc:648] HybridClock initialized: now 1786486862514524 us; error 91 us; skew 500 ppm
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:02.518900 10926 init.cc:377] Logged in from keytab as kudu/127.17.198.60@KRBTEST.COM (short username kudu)
I20260811 22:21:02.520499 10926 webserver.cc:460] Webserver started at http://127.17.198.60:45401/ using document root <none> and password file <none>
I20260811 22:21:02.521464 10926 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:02.521596 10926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:02.521982 10926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:02.525496 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data/instance:
uuid: "40135c3cc7d44d9f81a6eb4b02694fd6"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "fa270c6b61f8b6ee53ae10538a08467a"
server_key_iv: "4294ae0dce74abf2105c921a8db8f21a"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.526314 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal/instance:
uuid: "40135c3cc7d44d9f81a6eb4b02694fd6"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "fa270c6b61f8b6ee53ae10538a08467a"
server_key_iv: "4294ae0dce74abf2105c921a8db8f21a"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.533087 10926 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:21:02.537420 10945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:02.539381 10926 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:02.539700 10926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data,/tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal
uuid: "40135c3cc7d44d9f81a6eb4b02694fd6"
format_stamp: "Formatted at 2026-08-11 22:21:02 on dist-test-slave-77v9"
server_key: "fa270c6b61f8b6ee53ae10538a08467a"
server_key_iv: "4294ae0dce74abf2105c921a8db8f21a"
server_key_version: "encryptionkey@0"
I20260811 22:21:02.540033 10926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:02.562502 10926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:02.566989 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 3
I20260811 22:21:02.567479 10926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:02.567744 10926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:02.568300 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.567613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:36006)
0811 22:21:02.568077 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.568163 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324}
I20260811 22:21:02.569825 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.569420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:36022)
0811 22:21:02.569647 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.569722 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 22:21:02.570057 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.198.60:44455: Network error: Client connection negotiation failed: client connection to 127.17.198.60:44455: connect: Connection refused (error 111)
I20260811 22:21:02.580219 10926 rpc_server.cc:307] RPC server started. Bound to: 127.17.198.60:44455
I20260811 22:21:02.580600 11004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.198.60:44455 every 8 connection(s)
I20260811 22:21:02.581630 10926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/data/info.pb
I20260811 22:21:02.584015 10920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 3
I20260811 22:21:02.585081 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:02.585244 18200 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski7rXue/build/debug/bin/kudu as pid 10926
I20260811 22:21:02.585404 18200 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski7rXue/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486470476166-18200-0/minicluster-data/master-2/wal/instance
I20260811 22:21:02.585812 18200 external_mini_cluster.cc:1428] Setting key d00d26414bd29cc479843a79a0226c50
I20260811 22:21:02.590849 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.61@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:21:02.600311 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.584709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:36024)
0811 22:21:02.585005 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.585022 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.585142 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.588291 (+ 3149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.588295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.588555 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.588726 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.588748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.590433 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.590437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.591366 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.591382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.592502 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.592507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.592612 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.593409 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.593435 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.595001 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.597738 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.597741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.597743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.598272 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.598645 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.598652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.598654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.598756 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.599577 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.599581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.599720 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.599886 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.600084 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 22:21:02.601691 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.585853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36024 (local address 127.17.198.60:44455)
0811 22:21:02.586383 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.586405 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.586449 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:21:02.587440 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.587463 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.587911 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.588281 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.589116 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.590264 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.591552 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.592321 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.592941 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.595210 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.595216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.595231 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.595263 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.597575 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.598407 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.598412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.598417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.598537 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.598869 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.598872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.598874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.599400 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.599584 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.600048 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.600617 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":360,"spinlock_wait_cycles":55296,"thread_start_us":192,"threads_started":1}
I20260811 22:21:02.601691 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.586592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.17.198.60:44455)
0811 22:21:02.587018 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.587027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.587307 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:21:02.587440 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.587463 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.587911 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.588281 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.590305 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.590871 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.591801 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.592311 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.592484 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.596005 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.596015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.596018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.596079 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.599365 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.600043 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.600046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.600048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.600130 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.600478 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.600484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.600487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.600688 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.600925 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.601299 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.601413 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":341,"spinlock_wait_cycles":1280,"thread_start_us":122,"threads_started":1}
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:21:02.606972 10920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: Bootstrap starting.
I20260811 22:21:02.607267 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.594048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.17.198.62:42835)
0811 22:21:02.594207 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.594213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.594874 (+ 661us) server_negotiation.cc:407] Connection header received
0811 22:21:02.595044 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.595047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.595102 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.595175 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.596116 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.596996 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.598114 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.598561 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.598731 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.603076 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.603082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.603085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.603117 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.605294 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.605897 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.605900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.605902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.605993 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.606279 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.606283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.606285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.606506 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.606624 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.606955 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.607112 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:21:02.607291 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.593193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.62:42835 (local address 127.0.0.1:43030)
0811 22:21:02.594738 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.594787 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.594929 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.595326 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.595334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.595752 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.595956 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.595966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.597137 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.597141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.597977 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.597985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.598933 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.598938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.599110 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.600170 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.600189 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.602908 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.605424 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.605434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.605438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.605779 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.606105 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.606107 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.606108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.606181 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.606623 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.606630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.606750 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.606896 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.607096 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"mutex_wait_us":8,"thread_start_us":157,"threads_started":1}
I20260811 22:21:02.608422 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:02.610502 10920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:02.611364 10920 log.cc:824] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.62@KRBTEST.COM
I20260811 22:21:02.613410 10920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: No bootstrap required, opened a new log
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
I20260811 22:21:02.616664 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.604547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.17.198.62:42835)
0811 22:21:02.604946 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.604961 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.605601 (+ 640us) server_negotiation.cc:407] Connection header received
0811 22:21:02.605687 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.605690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.605751 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.605862 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.606607 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.607376 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.608460 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.608895 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.609355 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.612170 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.612174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.612177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.612206 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.614193 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.614728 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.614732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.614734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.614829 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.615141 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.615147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.615151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.615704 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.616219 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.616306 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.616435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":107,"threads_started":1}
I20260811 22:21:02.617606 10920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.617909 10920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d9766497ac4454bf5e34f25c2aec0a, State: Initialized, Role: FOLLOWER
I20260811 22:21:02.618537 10920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.618680 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.609071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.17.198.61:44035)
0811 22:21:02.609191 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.609196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.609356 (+ 160us) server_negotiation.cc:407] Connection header received
0811 22:21:02.609537 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.609541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.609593 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.609669 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.610063 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.610544 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.611642 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.612034 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.612116 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.615002 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.615005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.615008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.615032 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.616779 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.617312 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.617315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.617317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.617410 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.617736 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.617740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.617741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.618031 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.618132 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.618420 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.618543 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:21:02.618991 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.609024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.61:44035 (local address 127.0.0.1:53488)
0811 22:21:02.609270 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.609281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.609424 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.609701 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.609703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.609846 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.609940 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.609945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.610701 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.610703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.611492 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.611503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.612274 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.612277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.612394 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.613418 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.613436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.614885 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.616952 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.616955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.616957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.617211 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.617528 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.617531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.617533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.617628 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.618125 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.618128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.618242 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.618353 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.618496 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
I20260811 22:21:02.619017 11016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.619158 11016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.620020 10920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:02.620015 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.198.61@KRBTEST.COM
W20260811 22:21:02.629369 11029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:02.629489 11029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.60@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:21:02.627727 10842 master_runner.cc:186] Error getting master registration for 127.17.198.62:42835: OK, instance_id { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" instance_seqno: 1786486862091805 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:21:02.632251 10842 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:21:02.634712 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.618214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.17.198.61:44035)
0811 22:21:02.618855 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.618860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.618873 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:02.618928 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.618931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.618975 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.619056 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.619965 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.620680 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.621876 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.622168 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.622271 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.628950 (+ 6679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.628955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.628958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.628991 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.631779 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.632309 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.632315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.632317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.632410 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.632759 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.632763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.632765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.633068 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.633296 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.634277 (+ 981us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.634427 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":137,"threads_started":1}
I20260811 22:21:02.635167 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.620504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:36042)
0811 22:21:02.620695 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.620709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.620780 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.621322 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.621325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.621495 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.621579 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.621585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.623305 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.623312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.624257 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.624274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.626453 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.626457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.626605 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.629788 (+ 3183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.629808 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.631423 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.633570 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.633573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.633575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.633820 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.634089 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.634091 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.634093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.634169 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.634665 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.634668 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.634749 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.634857 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.634995 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":5}
I20260811 22:21:02.635756 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.620741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.17.198.60:44455)
0811 22:21:02.621053 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.621059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.621072 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:21:02.621118 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.621120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.621162 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.621245 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.621712 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.622951 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.625958 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.626324 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.626438 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.631562 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.631566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.631569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.631593 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.633446 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.633922 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.633924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.633926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.633995 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.634255 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.634257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.634259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.634538 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.634632 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.635491 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.635606 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":228}
I20260811 22:21:02.639566 18200 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:02.640255 11006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: Bootstrap starting.
I20260811 22:21:02.641167 10800 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2efa4d885dd04b46a2dc07b07629fdc4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43036
I20260811 22:21:02.643170 11006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:02.644110 11006 log.cc:824] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:02.647569 11006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: No bootstrap required, opened a new log
I20260811 22:21:02.652917 11006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.653240 11006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40135c3cc7d44d9f81a6eb4b02694fd6, State: Initialized, Role: FOLLOWER
I20260811 22:21:02.653622 11006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.654244 11033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.654534 11033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.655296 11006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:21:02.661005 11045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:02.661093 11045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:02.659706 10926 master_runner.cc:186] Error getting master registration for 127.17.198.62:42835: OK, instance_id { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" instance_seqno: 1786486862091805 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:21:02.663864 10926 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:21:02.734249 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } attempt: 4
Aug 11 22:21:02 dist-test-slave-77v9 krb5kdc[10736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.198.62@KRBTEST.COM for kudu/127.17.198.60@KRBTEST.COM
I20260811 22:21:02.747385 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.735729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.17.198.60:44455)
0811 22:21:02.735871 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.735877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.735893 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:21:02.735946 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.735949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.736001 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.736089 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.736799 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.737733 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.739022 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.739434 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.739544 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.742768 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.742773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.742776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.742814 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.744730 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.745386 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.745391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.745393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.745514 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.745887 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.745891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.745893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.746165 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.746296 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.747045 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.747189 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:21:02.748185 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.734974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.198.60:44455 (local address 127.0.0.1:36044)
0811 22:21:02.735378 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:02.735398 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:02.735509 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:02.736240 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:02.736243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:02.736516 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:02.736642 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.736650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.737893 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.737897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.738855 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:02.738868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.739699 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.739703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:02.739799 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.740438 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:02.740456 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:02.742540 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:02.744930 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.744936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.744938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.745250 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.745612 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:02.745616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:02.745618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:02.745717 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:02.746349 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:02.746353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:02.746477 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 22:21:02.746623 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 22:21:02.746804 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
I20260811 22:21:02.752506 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: Bootstrap starting.
I20260811 22:21:02.755584 10832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:02.756567 10832 log.cc:824] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:02.759296 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: No bootstrap required, opened a new log
I20260811 22:21:02.762668 10832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.762995 10832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2efa4d885dd04b46a2dc07b07629fdc4, State: Initialized, Role: FOLLOWER
I20260811 22:21:02.763355 10832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.764132 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.764493 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.765041 10832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:02.765107 10800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:21:02.765233 10800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:02.766860 10800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.767970 10800 leader_election.cc:288] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [CANDIDATE]: Term 1 election: Requested vote from peers 14d9766497ac4454bf5e34f25c2aec0a (127.17.198.61:44035), 40135c3cc7d44d9f81a6eb4b02694fd6 (127.17.198.60:44455)
I20260811 22:21:02.769548 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "14d9766497ac4454bf5e34f25c2aec0a"
I20260811 22:21:02.769829 10890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:02.771341 10890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efa4d885dd04b46a2dc07b07629fdc4 in term 1.
W20260811 22:21:02.771443 11059 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:02.771543 11059 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:02.771955 10974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6"
I20260811 22:21:02.772253 10974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:02.773795 10974 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efa4d885dd04b46a2dc07b07629fdc4 in term 1.
I20260811 22:21:02.776233 10776 leader_election.cc:302] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2efa4d885dd04b46a2dc07b07629fdc4, 40135c3cc7d44d9f81a6eb4b02694fd6; no voters:
I20260811 22:21:02.783491 10751 master_runner.cc:263] No leader master found from master 2efa4d885dd04b46a2dc07b07629fdc4
I20260811 22:21:02.783619 10751 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:21:02.783824 11048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:02.784205 11048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [term 1 LEADER]: Becoming Leader. State: Replica: 2efa4d885dd04b46a2dc07b07629fdc4, State: Running, Role: LEADER
I20260811 22:21:02.784662 11048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } }
I20260811 22:21:02.787743 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efa4d885dd04b46a2dc07b07629fdc4. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.787940 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:02.818012 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.788552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.17.198.60:44455)
0811 22:21:02.788706 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.788712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.793180 (+ 4468us) server_negotiation.cc:407] Connection header received
0811 22:21:02.793435 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.793439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.793511 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.793574 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.795115 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.796045 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.799356 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.799782 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.799884 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.809680 (+ 9796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.809687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.809691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.809727 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.811781 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.813662 (+ 1881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.813666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.813668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.813775 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.815472 (+ 1697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.815476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.815479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.815758 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.815937 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.817647 (+ 1710us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.817806 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:21:02.819716 10865 tablet.cc:2369] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:02.820243 10951 tablet.cc:2369] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:02.828410 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.789613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.17.198.62:42835)
0811 22:21:02.789761 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.789767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.795002 (+ 5235us) server_negotiation.cc:407] Connection header received
0811 22:21:02.797659 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.797663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.797728 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.797786 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.800776 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.801730 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.808238 (+ 6508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.808643 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.808748 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.820905 (+ 12157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.820910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.820914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.820953 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.823128 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.823857 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.823861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.823863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.823981 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.824356 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.824360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.824362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.824637 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.824779 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.828059 (+ 3280us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.828221 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:21:02.831019 11072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:02.832233 11072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:02.836248 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:02.790306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.17.198.61:44035)
0811 22:21:02.790466 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.790471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.799258 (+ 8787us) server_negotiation.cc:407] Connection header received
0811 22:21:02.803298 (+ 4040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.803304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.803372 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.803433 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.806141 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.806966 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.814179 (+ 7213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:02.814600 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:02.814705 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:02.825289 (+ 10584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:02.825294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:02.825297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:02.825332 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:02.827362 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.830196 (+ 2834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.830201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.830203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.830312 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:02.834912 (+ 4600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:02.834917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:02.834919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:02.835235 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:02.835405 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:02.835890 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 22:21:02.836057 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 22:21:02.839200 10778 tablet.cc:2369] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:02.848091 10974 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [term 1 FOLLOWER]: Refusing update from remote peer 2efa4d885dd04b46a2dc07b07629fdc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:02.849027 11048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:02.857196 11033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efa4d885dd04b46a2dc07b07629fdc4. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.857352 11033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.859174 10890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [term 1 FOLLOWER]: Refusing update from remote peer 2efa4d885dd04b46a2dc07b07629fdc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:02.859544 11066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.859695 11066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:02.859797 11066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:02.860687 11073 mvcc.cc:205] Tried to move back new op lower bound from 7317450190202912768 to 7317450189973643264. Current Snapshot: MvccSnapshot[applied={T|T < 7317450190202912768}]
I20260811 22:21:02.861550 11033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.861667 11033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.864516 11016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efa4d885dd04b46a2dc07b07629fdc4. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.864642 11016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.864729 11016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.864789 11016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:02.865402 11075 mvcc.cc:205] Tried to move back new op lower bound from 7317450190202912768 to 7317450189973643264. Current Snapshot: MvccSnapshot[applied={T|T < 7317450190202912768}]
I20260811 22:21:02.868083 11072 catalog_manager.cc:1347] Generated new cluster ID: 08882594f9e34ca19b7272237c8138c0
I20260811 22:21:02.868167 11072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:02.869695 11078 mvcc.cc:205] Tried to move back new op lower bound from 7317450190202912768 to 7317450189973643264. Current Snapshot: MvccSnapshot[applied={T|T < 7317450190202912768}]
I20260811 22:21:02.881314 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efa4d885dd04b46a2dc07b07629fdc4" member_type: VOTER last_known_addr { host: "127.17.198.62" port: 42835 } } peers { permanent_uuid: "14d9766497ac4454bf5e34f25c2aec0a" member_type: VOTER last_known_addr { host: "127.17.198.61" port: 44035 } } peers { permanent_uuid: "40135c3cc7d44d9f81a6eb4b02694fd6" member_type: VOTER last_known_addr { host: "127.17.198.60" port: 44455 } } }
I20260811 22:21:02.881495 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:02.885152 11072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:02.886548 11072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:02.896098 11072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2efa4d885dd04b46a2dc07b07629fdc4: Generated new TSK 0
I20260811 22:21:02.897025 11072 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:21:02.926939 11013 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.925249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.17.198.62:42835)
0811 22:21:02.925417 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.925423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.925617 (+ 194us) server_negotiation.cc:407] Connection header received
0811 22:21:02.925842 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.925858 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.925918 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.925988 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.926707 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43056: BlockingRecv error: recv got EOF from 127.0.0.1:43056 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:21:02.927084 11008 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.925050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.17.198.60:44455)
0811 22:21:02.925238 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.925244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.926001 (+ 757us) server_negotiation.cc:407] Connection header received
0811 22:21:02.926128 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.926134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.926193 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.926274 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.926929 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36056: BlockingRecv error: recv got EOF from 127.0.0.1:36056 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:21:02.927934 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.925705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.17.198.61:44035)
0811 22:21:02.925918 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.925924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.926357 (+ 433us) server_negotiation.cc:407] Connection header received
0811 22:21:02.926567 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.926572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.926632 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.926689 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.927792 (+ 1103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53518: BlockingRecv error: recv got EOF from 127.0.0.1:53518 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:21:02.931226 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.930140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.17.198.61:44035)
0811 22:21:02.930250 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.930255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.930341 (+ 86us) server_negotiation.cc:407] Connection header received
0811 22:21:02.930551 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.930553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.930619 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.930712 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.931117 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53520: BlockingRecv error: recv got EOF from 127.0.0.1:53520 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:21:02.935362 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.934456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.17.198.61:44035)
0811 22:21:02.934606 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.934611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.934625 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:21:02.934675 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.934678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.934735 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.934823 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.935246 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53526: BlockingRecv error: recv got EOF from 127.0.0.1:53526 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:21:02.940510 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.939406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.17.198.61:44035)
0811 22:21:02.939541 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.939546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.939673 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:21:02.939769 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.939772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.939831 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.939942 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.940387 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53532: BlockingRecv error: recv got EOF from 127.0.0.1:53532 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:21:02.948073 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.946550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.17.198.61:44035)
0811 22:21:02.946765 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.946769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.946943 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:21:02.947077 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.947079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.947127 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.947193 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.947909 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53540: BlockingRecv error: recv got EOF from 127.0.0.1:53540 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260811 22:21:02.959753 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.958259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.17.198.61:44035)
0811 22:21:02.958443 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.958451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.958687 (+ 236us) server_negotiation.cc:407] Connection header received
0811 22:21:02.958845 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.958848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.958904 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.958988 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.959579 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53546: BlockingRecv error: recv got EOF from 127.0.0.1:53546 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:21:02.979719 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:02.978250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.17.198.61:44035)
0811 22:21:02.978419 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:02.978425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:02.978627 (+ 202us) server_negotiation.cc:407] Connection header received
0811 22:21:02.978744 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:02.978747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:02.978803 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:02.978888 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:02.979550 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53556: BlockingRecv error: recv got EOF from 127.0.0.1:53556 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:21:03.015834 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:03.014451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.17.198.61:44035)
0811 22:21:03.014636 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.014639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.014745 (+ 106us) server_negotiation.cc:407] Connection header received
0811 22:21:03.014886 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.014889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.014940 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.015019 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:03.015686 (+ 667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53566: BlockingRecv error: recv got EOF from 127.0.0.1:53566 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 22:21:03.083483 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:03.082154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.17.198.61:44035)
0811 22:21:03.082334 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.082339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.082508 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:21:03.082699 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.082708 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.082758 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.082856 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:03.083332 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53570: BlockingRecv error: recv got EOF from 127.0.0.1:53570 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 22:21:03.215121 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:03.213901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.17.198.61:44035)
0811 22:21:03.214093 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.214096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.214177 (+ 81us) server_negotiation.cc:407] Connection header received
0811 22:21:03.214308 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.214311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.214360 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.214437 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:03.214970 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53586: BlockingRecv error: recv got EOF from 127.0.0.1:53586 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 22:21:03.475142 10924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:21:03.473750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.17.198.61:44035)
0811 22:21:03.474088 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.474092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.474112 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:21:03.474179 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.474182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.474244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.474343 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:03.474985 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53588: BlockingRecv error: recv got EOF from 127.0.0.1:53588 (error 108)
Metrics: {"server-negotiator.queue_time_us":244}
I20260811 22:21:03.630978 11029 catalog_manager.cc:1259] Loaded cluster ID: 08882594f9e34ca19b7272237c8138c0
I20260811 22:21:03.631084 11029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: loading cluster ID for follower catalog manager: success
I20260811 22:21:03.632766 11029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: acquiring CA information for follower catalog manager: success
I20260811 22:21:03.634078 11029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14d9766497ac4454bf5e34f25c2aec0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:03.662549 11045 catalog_manager.cc:1259] Loaded cluster ID: 08882594f9e34ca19b7272237c8138c0
I20260811 22:21:03.662652 11045 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: loading cluster ID for follower catalog manager: success
I20260811 22:21:03.664433 11045 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: acquiring CA information for follower catalog manager: success
I20260811 22:21:03.665889 11045 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 40135c3cc7d44d9f81a6eb4b02694fd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:03.994566 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:03.989736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.17.198.61:44035)
0811 22:21:03.990044 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.990047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.990123 (+ 76us) server_negotiation.cc:407] Connection header received
0811 22:21:03.990190 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.990193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.990329 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.990435 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:03.990687 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:03.991318 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:03.992359 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:03.992608 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:03.992699 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:03.993272 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:03.993784 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:03.993907 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:03.994339 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 22:21:03.994418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":102,"threads_started":1}
I20260811 22:21:04.003525 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:03.999092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.17.198.60:44455)
0811 22:21:03.999363 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:03.999367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:03.999383 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:21:03.999480 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:03.999482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:03.999616 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:03.999740 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:04.000242 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.000985 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.001935 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.002171 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.002289 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:04.002602 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:04.003126 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:04.003229 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:04.003325 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:21:04.003385 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
I20260811 22:21:04.012378 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:04.007792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53608 (local address 127.17.198.61:44035)
0811 22:21:04.007928 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:04.007933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:04.008371 (+ 438us) server_negotiation.cc:407] Connection header received
0811 22:21:04.008527 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:04.008532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:04.008591 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:04.008661 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:04.009050 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.009885 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.011118 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.011461 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.011559 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:04.011802 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:04.011916 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:04.011990 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:04.012154 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:21:04.012219 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:04.012845 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:04.007623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.17.198.60:44455)
0811 22:21:04.007772 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:04.007777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:04.008232 (+ 455us) server_negotiation.cc:407] Connection header received
0811 22:21:04.008446 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:04.008450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:04.008528 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:04.008580 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:04.009145 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.009927 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.010919 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.011224 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.011328 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:04.011907 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:04.011992 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:04.012076 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:04.012614 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 22:21:04.012675 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:21:04.013211 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:04.007488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.17.198.62:42835)
0811 22:21:04.007956 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:04.007992 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:04.008014 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:21:04.008133 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:04.008137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:04.008364 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:04.008563 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:04.008795 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.009571 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.010973 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:04.011343 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:04.011447 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:04.011877 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:04.012478 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:04.012934 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:04.012973 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 22:21:04.013041 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":93,"threads_started":1}
I20260811 22:21:04.021852 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10751
I20260811 22:21:04.031915 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10842
I20260811 22:21:04.041337 18200 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski7rXue/build/debug/bin/kudu with pid 10926
2026-08-11T22:21:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3475 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6906 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (393558 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestEncryptionWithKMSIntegration
2 FAILED TESTS